Matt's Past SAT/ACT News Update:

Matt O'Connor

Aug 10, 2018

 
Significant controversy has erupted over surprisingly low scores on the mathematics portion of the SAT administered in June. Due to an unusually easy test, many students have scored considerably lower than expected, in some cases despite answering more questions correctly than they did on past tests, on which higher scores were attained. Many students and parents have called for the test to be "re-scored" or "re-curved", but the anomaly arose not because of errors in such processes, but due to the composition of the math portion of the June SAT itself.

From the Washington Post article linked above:

[Excerpts]

...students argued the College Board should not have administered a test that varied so much in difficulty compared to other versions.

Marguerite Saunders, 17, said she answered 51 of 58 questions correctly on the math portion of the exam in March and received a 740. In June, she said she successfully answered 54 out of 58 questions and received a 700.

“It’s not the most accurate representation of my math ability and the whole reason people take the SAT is to have an accurate representation,” she said.

Saunders, a rising high school senior in the District, said she doesn’t disagree with the College Board’s use of equating but questioned “why they would release the test in the first place.”

Scott Jaschik collected several statements from parents and students online expressing confusion and anger over the SAT scoring results:

"College Board! One daughter got 760 getting only 5 wrong in math in March. Her twin missed 6 in math on June 2 test and got 670? 90 point difference in overall SAT scores for just ONE math question? How is that fair or standardized? So many kids hurt," wrote one parent.

"My first SAT I miss 26 questions and score a 1400. My second SAT I miss 16 questions and score a 1350?!?!" wrote one student.

So many comments have been made by students and parents about the disappointing June SAT math scores that the College Board has written a FAQ page about the issue.

[Excerpts]

Questions on tests administered on different dates are unique. Because of the difference between tests, you can’t directly compare the number of questions answered correctly. The equating process adjusts for the variation in difficulty between them.

...the SAT is not scored on a curve. Through equating, adjustments are made so that when a test is easier more questions need to be answered correctly. The reverse is also true. When a test is more difficult fewer questions need to be answered correctly to achieve the same reported score.

The Princeton Review weighs in on the June SAT scoring controversy by explaining, "Why You Don’t Want an 'Easy' SAT":

[Excerpts]

College Board took over the responsibility for writing the SAT from ETS when they redesigned the exam. The roll-out, as Reuters noted in a series of stories on the new SAT, has not always been smooth, particularly with respect to the math section of the exam:

In a nearly 5,000-word letter from August 2014, one reviewer told College Board officials that he had “never encountered so many seriously flawed items” in the 20-plus years he had been screening math material for the organization. (Reuters, 2016)

Despite these early jitters, things seemed more or less fine with the new test’s Math sections. Until today.

June 2018 SAT scores came out, and students took to Reddit to decry the Math curve for the exam. Students who got fewer questions wrong on the test than on previous attempts woke up to lower Math scores.

It’s worth noting that the College Board’s own difficulty ratings for questions, found in the Question and Answer Service (QAS) and Student Answer Service (SAS) forms that students can order for their tests have been very unreliable for the revised SAT. Their ratings should be, in a word, ignored.

Score equating is done before the test is ever given, so it’s worth saying that the actual performances on test day did not affect the curve. College Board knew it was going to administer an easier test, which meant more students would get more questions right, and the scale would need to undergo adjustment. As a result, small differences had a larger impact than usual.

...the equating applied to the June 2018 SAT suggests that the College Board made the test far too easy to distinguish among high scorers who received a score of 600 (76th percentile) or higher. That is a problem for those colleges who treat a 600, 650, a 700, a 750, and an 800 as accurate indicators of real differences in Math ability.

Compass Prep offers a detailed look at equating scores on the SAT in relation to the June controversy.

[Excerpts]

...when looking at the highest score that ever produced a 650 on the old [Math] SAT — where we have almost 50 released exams over 10 years — the lowest raw score was 40. The highest raw score to produce a 650 was 44. The curves never varied by more than 40 or 50 scaled points for the same raw score anywhere on the exams.

Compare this to how the June SAT 2018 Math fits in among its fellow new SATs. A 650 could be achieved with 50 correct answers. That’s the lowest scaled score the new SAT has ever produced for 50 correct answers. The highest score it has produced for 50 correct answers on an actual, released exam is 740 points — a 90-point swing! So in its first two years, the new SAT has approximately doubled the extremes seen on the old SAT over 10 years and 4 times as many exams. In terms of standard deviation, the June 2018 test was a full 2 SD further away from the mean than any other exam. When a 100-year flood occurs after two years, you have to be highly suspect of the weather forecasting.

College Board keeps coming back to the fact that the items were developed according to standards, the equating was computed correctly, the items were scored correctly, and the resultant scale was correct. There is no direct evidence that College Board misscored the exams. There is, however, evidence that College Board issued an exam that it would have known was well out of spec. It is expensive to throw away an exam, but that’s exactly what College Board should have done.

The other major anomaly with the Math exam was the set of gaps between scaled scores — especially at the high end. On the June SAT, no one received a 710, a 730, a 740, a 760, a 780, or a 790; over half of the possible scores at or above 700 went unused. A single wrong answer could have dropped a student by as many as 30 points. There were surely test takers who would have normally filled those gaps, but the June SAT was incapable of sorting them properly. The over-reliance on easy questions did not give the SAT enough power to make fine distinctions among high-scoring students. And while admissions officers are cautioned not to make meaningful distinctions between small score differences, highly selective colleges are flooded with scores in this range and may perceive contrast that the tests themselves can’t see.

The incongruity of the June scale is a valid criticism. Indeed, College Board includes “minimized gaps” among its own goals of test construction in its SAT Technical Manual. No other new SAT has anywhere close to that many gaps. Some have none at the high end.

...concerns grow as stakeholders — students, parents, counselors, and even colleges — question if the SAT is losing its way. Compass is quick to defend the SAT when it is wrongly blamed, but we’ve also written about its owner’s missteps which have seemed to increase since 2012 when current leadership assumed power. Significant internal changes have occurred in recent years, including the loss of key members of the SAT team and the reclaiming of test construction from ETS, which had long been subcontracted to create tests. It seems College Board may be learning the hard way how difficult the job really is.

In addition to the relatively easy June SAT math test, the College Board also decided not to score 4 questions that appeared on the June SAT test form (2 reading and 2 writing), due to scoring anomalies that raised concerns about the fairness of the questions.

 
Furthering a recent trend that has seen elite US universities such as Harvard, Princeton, and Stanford drop the requirement for applicants to submit SAT essay or ACT writing scores, Brown University, Duke University, CalTech, and the University of Michigan have now announced that the submission of such scores will be optional in future admissions cycles.

Among the eight Ivy League universities, Brown had been the last holdout requiring the essay/writing test scores. Brown University’s dean of admission Logan Powell addressed the rationale for the policy change:

“Given the significant growth in free school-day testing, it’s important to enable students from low-income families to take advantage of the tests already offered by their school districts and not place an undue burden on them to go in separately outside of normal school hours. Our goal is that for any talented student interested in Brown, the application process is not a deterrent — and we don’t want this test to be a barrier to their application.”

At Duke, the submission of SAT essay/ACT writing scores will be optional, but recommended. Christoph Guttentag, Duke's dean of undergraduate admissions, commented:

“We will continue to value writing as particularly meaningful as we develop a sense of students as potential members of the Duke community. And we will still pay careful attention to essay scores and what they represent for those students who submit them. We also recognize that this part of the exam can represent more of a barrier for some students than others, and we want to give every student an opportunity to be fully considered in our application process.”

Motivation for the CalTech decision was provided by a statement from the university stating,

"Writing and communications skills are valued highly by Caltech and will continue to be evaluated through the information collected in the SAT/ACT verbal sections as well as through required application essays. With this policy, Caltech aims to streamline the application process and eliminate additional testing fees incurred by applicants."

In an e-mail to Inside Higher Ed's Scott Jaschik, a spokesman for the University of Michigan explained the policy change:

"Several factors went into this decision, including the recent redesign of the evidence-based reading and writing section of the SAT that incorporates a more expansive reading and writing assessment within the body of the exam, as well as our continued requirement that applicants already submit multiple writing samples to our UM-specific admissions application."

 
When considering the spate of recent decisions at elite institutions to drop the essay and writing requirements, Scott Jaschik writes "For Fate of SAT Writing Test, Watch California".

[Excerpts]

For those hoping that that the essay disappears, there may be one giant obstacle: the University of California. The system's undergraduate campuses receive more than 200,000 applications per year, and all applicants must complete the SAT or ACT essay. The College Board has given every indication over the years that it sees the University of California as a crucial system when it comes to any changes in the SAT. A 2001 speech by Richard Atkinson, then president of the system, led the College Board to create the writing test, which debuted in 2005.

While faculty members have been involved in some colleges' decisions to drop the essay, the University of California decision is controlled by a systemwide faculty body, the Board of Admissions and Relations with Schools, known by its acronym BOARS.

That committee has been "monitoring" the decisions of other colleges and has been studying whether UC should maintain the requirement, said Henry Sanchez, a professor of pathology at the University of California, San Francisco, and chair of BOARS. "We are looking at this."

He noted that UC has "a different situation" from some of the colleges dropping the requirement in the vast number of applications that the system receives, making a standardized option appealing.

BOARS will consider the issue over the next year, in what he said would be "a very faculty-driven, evidence-based analysis."

In the wake of the University of Chicago's decision to make the submission of SAT and ACT scores optional by applicants, an opinion piece in The Stanford Daily asks "Should Stanford go test-optional?"

[Excerpts]

As for the possibility that Stanford drops their test score requirement entirely, Mike Devlin, Stanford’s Associate Dean and Director of Admission, says that Stanford will not be following in UChicago’s footsteps any time soon.

“As for Stanford, we will continue to require the SAT or ACT from all applicants,” Devlin wrote in an email to the Daily. “Standardized testing will be one factor among many that we consider during our review process. We do not intend to alter this requirement now or in the foreseeable future.”

Five more colleges and universities have announced test optional admissions policy adoptions:

Ball State University (US News-ranked #187 among National Universities; 17,000 undergraduates enrolled; 62% acceptance rate; 990-1170 25th/75th SAT scores) has announced the adoption of a test optional policy.

Framingham State University (US News-ranked # 126 among Regional Universities North; 4,300 enrolled undergraduates; 65% acceptance rate; 890-1090 25th/75th SAT scores) has announced that has “joined a Department of Higher Education Pilot Program that allows certain students to opt out of sharing their SAT or ACT scores.”

Pacific Lutheran University (US News-ranked #16 in Regional Universities West; 2,800 undergraduates enrolled; 77% acceptance rate; 980-1230 25th /75th SAT scores) has also announced the adoption of a test optional policy.

Sweet Briar College (US News-ranked # 134 among National Liberal Arts Colleges; 365 undergraduates enrolled; 93% acceptance rate; 880-1180 25th /75th SAT scores) now offers test optional admissions consideration for students with a high school GPA of 3.0 or higher.

The University of New England (US News-ranked # 78 in Regional Universities North; 4,300 undergraduates enrolled; 83% acceptance rate; 940-1150 25th /75th SAT scores) has also gone test optional.