Posts

ACT Explains their Essay Scores

We’ve talked about the new ACT essay and its associated issues and confusions. ACT has released a document hoping to clear up the misconceptions, titled “ACT Research Explains New ACT Test Writing Scores and Their Relationship to Other Test Scores.”

I don’t want to ruin this delightful saga for you, but you should know going on: there is no relationship. Beyond that, ACT says there’s no relationship between individual subject scores, either.

ACT addresses the discrepancy between the Writing scores and the other scores by acknowledging that there is a difference in scores, but that concern about that fact is rooted in a lack of communication about how scores should be understood.

 

“It is true that scores on the writing test were on average 3 or more points lower than the Composite and English scores for the same percentile rank during September and October 2016. Some students may have had even larger differences between scores. This is not unexpected or an indication of a problem with the test. However, the expectation that the same or similar scores across ACT tests indicate the same or similar level of performance does signal that ACT needs to better communicate what test scores mean and how to appropriately interpret the scores.” 

Much of the article reads as an attempt by ACT to debunk the idea that their individual scores are very useful. For example, “the difference in scores across tests does not provide a basis for evaluating strengths and weaknesses….” The important thing to look at, they maintain, are the percentiles, because scores don’t really translate across subject lines. A 25 in Reading is not a 25 in Math, and you were silly to think, because they’re reported on the same scale and created by the same people, that there was any relationship between them. (OK, so they don’t actually call anyone silly. But they do say that “[t]ests are not designed to ensure that a score of 25 means the same thing on ACT Math as it does for ACT Science or Reading.” Check out that skilled usage of the passive voice there!)

There have always been some differences between the scores in different sections. Anyone who looks at a percentile chart will notice that. For the most part, they’re not huge.  But “even larger differences are found when comparing percentiles between the new ACT subject-level writing score and the other ACT scores. For example, a score of 30 on the ACT writing test places that same student at the 98th percentile, a full 9% higher than the reading score [of 30.] Similarly, an ACT subject-level writing score of 22 is over about 10 percentiles about the Composite or other ACT scores.”

Why change over to the new scale at all then? Don’t you think that the new scale might, you know, encourage those comparisons? The ACT admits the 1-36 score for the Writing “[makes] comparisons with the other scores much more tempting. Perhaps too tempting!”

ACT’s insistence on talking about the test like something that sprang into being all by itself and behaves in self-determining ways is giving me robot overlord concerns.

 

Didn’t they… design it? Didn’t they decide how the scoring would work? Unless the ACT test itself has become sentient and begun making demands, this doesn’t make any sense. Treating the scores as if they exist as something other than a product of a series of human decisions is absurd.

They put everything on the same scale. They’ve admitted that having things on an apparently identical scale invites comparison, and they’ve also said that “[c]omparisons of ACT scores across different tests should not be made on the basis of the actual score…” Oh, OK. What?

If the actual scores don’t really help in making comparisons (which is what standardized tests are for, after all – making comparisons), and everyone should just look at the percentiles, then why do we need the 1-36 scale at all?

In the document, ACT points out multiple times that this kind of variation is common on many tests. That doesn’t change the fact that they changed the Writing score to make it look like the other scores (except lower) which has caused confusion and will continue to do so. The math that turns an 8 into a 23 and a 9 into a 30 is not intuitive nor set in stone, and if they wanted to correlate a given Writing score and a Math or English score to the same percentile, they could.

Beyond that, as the ACT admits, “each test score includes some level of imprecision,” and that margin of error is considerably larger for the Writing test. They explain that “a score of 20 on ACT writing would indicate that there is a two-out-of-three chance that the student’s true score would be between 16 and 24.” (Put another way, this means there is a 33% chance that your “true score,” whatever that means, is more than 4 points away from what shows up on your report.”) The ACT cautions against using the Writing score on its own to make decisions, suggesting using the ELA score instead – an average of Reading, English and Writing. But if the Writing score on its own isn’t a useful metric, then perhaps it shouldn’t be reported?

Further complicating all of this, the ACT blames some of the low scoring on students’ unfamiliarity with the new version of the test: “students are only beginning to get experience with the new writing prompt. Research suggests that as students become increasingly familiar with the new prompt, scores may increase[.]”

The ACT claims that the essay tests “argumentative writing skills that are essential for college and career success.” If that were actually the case, familiarity with the test wouldn’t matter. Unless the ACT is arguing that as students become more familiar with this particular aspect of the test, they will also become more prepared for college. I suggest that those two factors – college readiness and ACT essay scores – don’t have a relationship, either.

Author: Audrey Hazzard, Master-Level Tutor

ACT Essay Score Issues

Have you noticed a discrepancy between your ACT multiple-choice scores and your Writing score? You’re not alone. Since the new essay debuted in September, students have been frustrated and alarmed by low essay scores. The concordance between the old and new essay scores equates an old 8/12 with a new 23/36, but an old 9/12 with a 30/36 – a huge gap. Many students who are scoring in the 30s on their multiple choice sections are seeing essay scores in the low to mid 20s, and they are understandably concerned.

The Washington Post spoke to the parents of a student who took the ACT in September. He received “a 19 on the writing section and 30s on the rest of the test.” When the student requested a rescore of his essay, the score increased – to a 31. Based on the new essay scoring, that jump – from a 19 to a 31 – represents a change from the 63rd percentile to the 98th.

Wait, you’re thinking, a 19 is the 63rd percentile? Yes, on the Writing, it is. For comparison, a 19 on English is the 45th percentile. On the Science section, a 19 is in the 40th.

Top Tier Admissions wrote of the new ACT essay scoring, “Imagine a teacher giving a test where a 70% was the highest score out of a thousand students, but then deciding not to curve the test. That is what is happening right now on this new writing section.”

Criticism of the new ACT essay has been widespread, so if you’re feeling concern or even shock about your essay score, know that you’re not alone. Resist the temptation to compare your 1-36 essay scores with the 1-36 scores in the other sections: look at the percentiles, instead. If you’re considering retaking the ACT just to boost your essay, check in with your admissions reps at your top-choice colleges first. More colleges, including Tufts, Penn, Brown, and Swarthmore, are opting not to require the essay portion of the exam at all.

Even if your college does require it, it may not be worth retaking the whole exam if you’re pleased with the rest of your scores. Talking with your admissions representative may help you determine how important that score is – or isn’t.

Changes to the ACT

ACT recently unveiled plans for a 2015 update to their test. Students taking the ACT next year will need to prepare for a more complex essay, as well as some minor alterations to the Reading and Mathematics tests. Relative to the SAT’s extensive changes, the changes to the ACT will be subtle.

Writing:

    • For students taking the optional ACT Writing portion, the writing prompt may become more complex, requiring students to “evaluate multiple perspectives on a complex issue and generate their own analysis based on reasoning, knowledge, and experience.” Although the prompt may appear to be more challenging, the additional direction in the prompt and hypothetical perspectives provided will allow the student to actively analyze stated ideas, instead of initializing the entire topic.  In addition, the student will be provided with an additional ten minutes to write the essay – for a total of 40 minutes.
    • The scoring of the Writing test will be also updated and will include subscores in four areas: ideas and analysis, development and support, organization, and language use.

Reading:

    • The Reading test will introduce a new passage type. Students will be asked to compare information from paired passages (dual passages), similar to a subset of questions on the SAT. The ACT began rolling out these paired passages during the 2014-2015 school year, but each test will have this dual passage from this point forward.

Mathematics:

    • The Mathematics section will see a very slight increase of emphasis on statistics and probability in the Math test. The change will be minor enough that most students probably won’t notice the difference.

Science:

    • The Science section will have the same number of questions (40), but they will be broken down into 6 themed passages rather than 7.  We perceive this as giving each student a leg up, as now there are only six varied science related topics that must be addressed, rather than seven topics that students may only have limited previous knowledge of beforehand.

The main focus of the upcoming ACT update seems to be on state assessments, rather than on college applications. This is evidenced by the ACT’s new supplemental scores: STEM and Language Arts scores, and Career Readiness and Text Complexity indicators. There will also be new question categories aligned to Common Core standards. Most of these new scores are designed to provide more detailed insight into students’ progress.

The ACT’s traditional 1-36 composite score will not change; these new scores will be provided in addition to the current provided scores. Other changes affecting ACT state assessments include the addition of more optional tests (math, reading, and science) and increased availability of digital tests.

These changes to the ACT are planned to take effect in fall 2015 and in 2016.