In December, the New York Times had a "Room for Debate" piece called Why Does the SAT Endure? The viewpoints expressed include those of two psychometricians, a college admissions officer, someone working for a test-prep company, and an education policy wonk. Taken together, the pieces didn't constitute much of a debate, but the introduction to the discussion poses the question of why the SAT is still around if, as its critics say, it can be gamed.

No one really answered the question directly, but the implicit answer of everyone writing here was that the SAT remains widely used because it works as well as anything available. Most notably, both the psychometricians, who have the technical expertise to evaluate the research about the SAT, accept the SAT as a valid tool for college admissions.

As for the notion of "gaming" the test, everyone, with the exception of the test-prep guy, seemed to treat that as synonymous with paid test-preparation courses. He claimed that improvements came from learning real skills and weren't the result of quick tricks. None of the others discussed the content of test-prep courses. Their consensus was that test-prep courses can lead to modest gains, but they probably aren't worth the time and money for most students.

The question of test gaming is an important one, and deserves a more nuanced discussion than it received in the times. What exactly do we mean by gaming the test?

The NY Times piece was nominally provoked by the Long Island cheating scandal. Paying someone to take a test for you, along with other forms of overt cheating, is certainly gaming the test. And if cheating is too widespread, it does threaten test validity. But this is an external threat, a matter of security rather than an internal flaw inherent to the design of the test itself. Clearly, the high-stakes nature of the SAT is going to tempt some people to try to cheat on it, but cheating is not limited to standardized tests. Given the security measures that the SAT takes, it's probably fair to say that there is overall much less cheating on the SAT than there is on normal school evaluations, or on NCLB tests. For NCLB tests, you have the added problem of teachers and administrators collaborating with their students to cheat, as the adults are held accountable for the students' performance.

Cheating also has nothing to do with the normal activities of test-preparation companies. (Full disclosure: I work for one such company.) It is true that there have been cases of unethical companies that help their clients cheat. In 2007, for example, at least one hagwon in Korea gave its students answers to the January 2007 test. As a result, all the SAT scores from Korea were canceled for that form. This sort of test-leaking was possible because (a) the SAT regularly uses the same form twice, separated by a couple of years, (b) unethical test-prep companies have test takers steal test booklets, and give these tests to their students as practice.

The controversy over test-preparation, however, isn't about cheating, but about the methods these companies use to prepare students to take the test following all the regulations established by the College Board. So is test preparation in general "gaming" the test? Let's agree first on what it means to game the test.

The normal understanding of game used in this sense involves exploiting some inherent flaw in the system to get an advantage. Hence, Enron gamed the energy market in California by exploiting flaws in the market regulations. To game a test involves solving questions by using properties of the test that are neither part of the subject that is being tested nor an intentional part of the test design.

In other words, it is clearly not gaming the test to study the underlying content tested. This is what we want students to do, whether that happens in their ordinary classes or in specialized test-preparation classes. Learning college-level vocabulary, for example, will almost certainly improve your score on the critical reading section of the SAT. But since that's one of the things the critical reading section is designed to measure, that's a good thing. If your score goes up because your vocabulary is better, that's a real skill improvement.

Beyond that, it is not gaming the test to learn about the instructions for different test questions. Nor is it gaming the test to take practice tests where you work on applying what you know to solving representative problems in the given time. While these are specific to taking a test rather than the subject being tested, they are an inevitable part of test taking. The test has to be in some specific format, and we expect students to be familiar with the mechanics of test-taking.

So what is gaming the test? The clearest instances involve answering questions without solving them in any way that resembles the process the test makers assume you will use. For example, if you answer the reading questions without actually looking at the passage, you are clearly gaming the test.

Other strategies occupy a fuzzier borderland between gaming and direct effort. For example, on the writing section, if you always choose the shortest answer for sentence-revision questions (a simplified version of some common advice), you are gaming the test. You're taking advantage of a general principal that shorter is usually better without actually looking at the context. On the other hand, if you're comparing two answer choices that both look grammatically correct and then choosing the one, that expresses the same idea in fewer words, you are appropriately applying a widely accepted stylistic principal.

Are test-prep companies who teach students to prefer the shorter choice when all else appears equal engaging in gamesmanship? I don't think so.

Do some test-prep companies teach tricks that amount to gaming the test? Yes. But as I shall argue in the next part, gaming strategies are not nearly as effective as some test-prep companies claim. Does that mean that test-preparation isn't worthwhile. That would be true if gamesmanship was all that was taught I'll also take up the question of how effective test preparation is next time.