{"id":10288,"date":"2011-10-25T13:18:57","date_gmt":"2011-10-25T13:18:57","guid":{"rendered":"http:\/\/scienceblogs.com\/gregladen\/2011\/10\/25\/ooops-we-left-all-the-children\/"},"modified":"2011-10-25T13:18:57","modified_gmt":"2011-10-25T13:18:57","slug":"ooops-we-left-all-the-children","status":"publish","type":"post","link":"https:\/\/gregladen.com\/blog\/2011\/10\/25\/ooops-we-left-all-the-children\/","title":{"rendered":"Ooops&#8230; we left all the children behind"},"content":{"rendered":"<p>&#8220;No Child Left Behind&#8221; was doomed to be a failure, because it was ill concieved, politically cynical, and underfunded.  But like the War in Iraq, the Patriot Act and Tax Breaks for the Rich, and all the other initiatives of the Bush Administration that never should have happened, we have been saddled with this melt down of a policy for over a decade.  &#8220;NCLB&#8221; was one of the first policies implemented by Bush.<\/p>\n<p>The evidence that the approaches developed under this policy have failed has been mounting for years, and the supporters of NCLB have been dropping like flies on no-pest strip.  The latest and perhaps most important policy related statement to date has just come out, and it is a study published by the National Academies of Science:   <em>Incentives and Test-Based Accountability in Education<\/em> by Michael Hout and Stuart W. Elliott, Editors; Committee on Incentives and Test-Based Accountability in Public Education; National Research Council.<\/p>\n<p>And the study says &#8230;<br \/>\n<!--more--><\/p>\n<blockquote><p>In recent years there have been increasing efforts to use accountability systems based on large-scale tests of students as a mechanism for improving student achievement. The federal No Child Left Behind Act (NCLB) is a prominent example of such an effort&#8230; bla bla bla &#8230;  For the first time, research and theory on incentives from the fields of economics, psychology, and educational measurement have all been pulled together and synthesized. Incentives and Test-Based Accountability in Education will inform people about the motivation of educators and students and inform policy discussions about NCLB &#8230; bla bla bla &#8230;<\/p><\/blockquote>\n<p>Never mind that part &#8230; it&#8217;s just the text they put on the front page so as it arrives on the desks of the members of congress who implemented this policy they don&#8217;t have amateurism.  Let&#8217;s get right to the conclusions.  There are two:<\/p>\n<p><strong>Conclusion 1: Test-based incentive programs, as designed and implemented in the programs that have been carefully studied, have not increased student achievement enough to bring the United States close to the levels of the highest achieving countries. When evaluated using relevant low-stakes tests, which are less likely to be inflated by the incentives themselves, the overall effects on achievement tend to be small and are effectively zero for a number of programs. Even when evaluated using the tests attached to the incentives, a number of programs show only small effects. Programs in foreign countries that show larger effects are not clearly applicable in the U.S. context.  School-level incentives like those of the No Child Left Behind Act produce some of the larger estimates of achievement effects, with effect sizes around 0.08 standard deviations, but the measured effects to date tend to be concentrated in elementary grade mathematics and the effects are small compared to the improvements the nation hopes to achieve.<br \/>\n<\/strong><\/p>\n<p>Holy crap!  The policy was ideologically sound and based on free market theory and stuff!  But it didn&#8217;t work!<\/p>\n<p><strong>Conclusion 2: The evidence we have reviewed suggests that high school exit exam programs, as currently implemented in the United States, decrease the rate of high school graduation without increasing achievement. The best available estimate suggests a decrease of 2 percentage points when averaged over the population. In contrast, several experiments with providing incentives for graduation in the form of rewards, while keeping graduation standards constant, suggest that such incentives might be used to increase high school completion.<br \/>\n<\/strong><\/p>\n<p>OMG!  The No Child Left Behind Program has <em>RUINED<\/em> education in America instead of <em>FIXING<\/em> it!<\/p>\n<p>There are three recommendations and they are rather disappointing.<\/p>\n<p><strong>Recommendation 1: Despite using them for several decades, policy makers and educators do not yet know how to use test-based incentives to consistently generate positive effects on<br \/>\nachievement and to improve education. Policy makers should support the development and evaluation of promising new models that use test-based incentives in more sophisticated<br \/>\nways as one aspect of a richer accountability and improvement process. However, the modest success of incentive programs to date means that all use of test-based incentives should be carefully studied to help determine which forms of incentives are successful in education and which are not. Continued experimentation with test-based incentives should not displace investment in the development of other aspects of the education system that are important complements to the incentives themselves and likely to be necessary for incentives to be effective in improving education.<\/strong><\/p>\n<p><em>Translation:  What we are doing now does not work.  There are some efforts to do what we are doing now which have shown insufficient promise, so pursue those for a while but then quietly drop them and shift to the unmentionable things that would actually work.  Which we shall not mention. <\/em><\/p>\n<p><strong>Recommendation 2: Policy makers and researchers should design and evaluate new test-based incentive programs in ways that provide information about alternative approaches to incentives and accountability. This should include exploration of the effects of key features suggested by basic research, such as who is targeted for incentives; what performance measures are used; what consequences are attached to the performance measures<br \/>\nand how frequently they are used; what additional support and options are provided to schools, teachers, and students in their efforts to improve; and how incentives are framed and communicated. Choices among the options for some or all of these features are likely to be critical in determining which&#8211;if any&#8211;incentive programs are successful.<\/strong><\/p>\n<p><em>Translation: Stop teaching to the test, stop funding schools based on test results, stop designing systems that move the encouragement to cheat from the student up to the administrator, and instead implement other approaches that may actually work which we shall not name. <\/em><\/p>\n<p><strong>Recommendation 3: Research about the effects of incentive programs should fully document the structure of each program and should evaluate a broad range of outcomes. To avoid having their results determined by the score inflation that occurs in the high-stakes tests attached to the incentives, researchers should use low-stakes tests that do not mimic the high-stakes tests to evaluate how test-based incentives affect achievement. Other outcomes, such as later performance in education or work and dispositions related to education, are also important to study. To help explain why test-based incentives sometimes produce negative effects on achievement, researchers should collect data on changes in educational practice by the people who are affected by the incentives.<\/strong><\/p>\n<p><em>Translation: Go back to basics and find out what works.  Consider alternation methods that shall not be named.  Consider aligning the objectives of education with the realities of society.<\/em><\/p>\n<p>Now we just have to undue the fabricated belief that class size does not matter, strengthen efforts to professionalize the teaching profession, and address issues of variation among students in ways that allow us to work with what we&#8217;ve got rather than to ignore both problems and potentials among our students.<\/p>\n<p><a href=\"http:\/\/books.nap.edu\/catalog.php?record_id=12521&#038;utm_medium=etmail&#038;utm_source=National%20Academies%20Press&#038;utm_campaign=NAP+mail+new+10.25.11&#038;utm_content=&#038;utm_term=\">You can get your own copy of this report here. <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;No Child Left Behind&#8221; was doomed to be a failure, because it was ill concieved, politically cynical, and underfunded. But like the War in Iraq, the Patriot Act and Tax Breaks for the Rich, and all the other initiatives of the Bush Administration that never should have happened, we have been saddled with this melt &hellip; <a href=\"https:\/\/gregladen.com\/blog\/2011\/10\/25\/ooops-we-left-all-the-children\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Ooops&#8230; we left all the children behind<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"1","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[2030,183,4398,4399,4400],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p5fhV1-2FW","jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/posts\/10288"}],"collection":[{"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/comments?post=10288"}],"version-history":[{"count":0,"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/posts\/10288\/revisions"}],"wp:attachment":[{"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/media?parent=10288"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/categories?post=10288"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/gregladen.com\/blog\/wp-json\/wp\/v2\/tags?post=10288"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}