But she says the computer scoring has proven "spot-on" and Utah now lets machines be the sole judge of the vast majority of essays. Cyndee Carter, assessment development coordinator for the Utah State Board of Education, says the state began very cautiously, at first making sure every machine-graded essay was also read by a real person. Several states including Utah and Ohio already use automated grading on their standardized tests. The computer also breaks it down in several categories of sub-scores showing, for example, a one on spelling and grammar, and a two on task and focus. "It gives an overall score of two out of four," Foltz explains. To demonstrate, he takes a not-so-stellar sample essay, rife with spelling mistakes and sentence fragments, and runs it by the robo-grader, which instantly spits back a not-so-stellar score.
"We've done a number of studies to show that the scoring can be highly accurate," he says.
That includes not only basics like spelling and grammar, but also whether a student is on topic, the coherence or the flow of an argument, and the complexity of word choice and sentence structure. "We have artificial intelligence techniques which can judge anywhere from 50 to 100 features," Foltz says. Then, the automated programs score essays themselves by scanning for those same features. Foltz says computers "learn" what's considered good writing by analyzing essays graded by humans.
0 kommentar(er)
