2 Aug 2018

Jack-and-the-Beanstalk Test Prep

Submitted by Karl Hagen
I can't tell you how many students I've heard utter some variation of the following: "I took a practice test and my parents flipped out at how low the score was. I'm taking the test in [exceedingly close month], and I have to raise my score by at least [absurdly large amount]. I really need to raise my score fast."

This attitude reflects what I like to think of as the Jack-and-the-Beanstalk theory of test preparation. This is the notion that somewhere out there are magic beans that someone can sell you which will produce huge growth with minimal effort on your part. A lot of American students have this idea that standardized tests can somehow be "beat" with a few simple gimmicks. For the most part, that's all marketing hype from companies eager to sign you up for their courses. In the real world, though, the beans you've traded the cow for aren't magic. Even that high-end test prep which families shell out thousands of dollars for won't produce results unless you are willing to put in real work, and some of the increases those programs claim aren't the result of anything special that they're doing.

Here's the reality about test prep and score increases: When students start to study a particular test for the very first time, it's common to see noticeable early increases in their scores. (Note that I'm assuming a student who's done no prior prep and is getting scores from official practice tests, taken under timed conditions. If you're taking mock tests from a test-prep company you should put little if any weight on those scores. Leaving aside the issue of the quality of the material, they're almost never scaled adequately.)

This is what I call the "familiarization bump." Your score goes up as you become accustomed to the format of the questions, the timing, and other practical elements of the test. What you're doing here is picking low-hanging fruit. You're getting rid of the most obvious barriers that interfere with you applying what you already know to the particular format of the test. And while a teacher can speed up this process a little, you're likely to see this bump no matter how you study. This increase has little to do with the form of study you're putting in.

The familiarization bump is one of the things that makes it extremely difficult to evaluate the claims test-prep companies make about how much they've helped their students, and one reason that low-quality test-prep outfits can succeed. They get credit for increases that happen regardless of the program.

After that early, quick rise, scores normally go up much more slowly, and they often hit a plateau. This is the point where students typically become frustrated. If you're at this point, don't waste your time looking for the shortcut that will make your score rocket up. It doesn't exist. You'll improve by learning fundamental skills. This is particularly true if you're already scoring fairly high on your chosen test. Most test tricks are probabilistic strategies (e.g., guess the shortest answer on a grammar question), and while there's often some validity to them on average—that is, if you follow them you'll get more questions right than you miss—they are wrong often enough that you can't count on them to get an above-average score.

The people who get perfect scores don't get them by guessing. They read a complex passage and understand what both it and the questions mean because they know enough vocabulary to make sense of it, because they can follow intricate syntax, and because they've developed strong reading habits over the years. They understand the basic principles of carefully edited writing and can consciously apply that knowledge instead of just relying on what "sounds right." They have the math background to solve problems directly and accurately. To break out from whatever score you're stuck at, you will have to build those fundamental skills. If you're not putting in the real work required to do that, you're just going through the motions of preparing.

That may seem like a daunting prospect, but think of it this way: if you study for the SAT, the ACT, or any other entrance exam by working on your fundamental skills, you'll be able to use that later in your later academic career, and possibly even in life after school. If you focus on test-specific tricks you're spending your time on things that don't work very well and which will be useless to you after you're done with the test.