Bad Grades

"Last semester Lindsay Hutton "taught" 1,940 students. She met only 70 of them in person. Those were the ones enrolled in the two weekly sections of English composition that she taught in an actual classroom. The hundreds and hundreds of others she knew only as anonymous numbered documents she read on her computer screen and then, with a click of a button, sent back out into the ether.

As one of 60 graduate students hired to teach freshman composition at Texas Tech University, Ms. Hutton had a weekly quota of grading.

Each week she was assigned to read, comment on, and grade 17 drafts of essays, offer a second grade on 18 more, and review about 25 peer critiques and 20 student self-evaluations to fulfill her 12-hour grading responsibility Ð'-- allowing an average of 10 minutes per document. Ms. Hutton, a Ph.D. student in creative writing, is not paid per draft, but some graduate students who take on grading work during the holidays are. Last spring, for example, students were paid $2 for grading a preliminary draft, $4 for a final draft, and 50 cents for a peer or self-evaluation.

"Sometimes," Ms. Hutton says, "it feels like a factory."

Most colleges and universities require some kind of first-year writing course. Increasingly, the expensive task of getting masses of freshmen from widely varying educational backgrounds up to snuff falls to the cheapest labor: untenured professors, part-timers, and graduate students.

While institutions around the country are experimenting with technology to enhance or reinvigorate freshman comp, Texas Tech is using a computer system to entirely reinvent the experience. The university has cut class time in half, increased the amount of writing students do, and split the teachers into two groups: "classroom instructors" and "document instructors." The system allows faculty members to closely monitor the graders and collect piles of data about student writing and how graduate students evaluate it.

The system has divided the English department, pitting professors who say it not only saves time but prevents biased grading against those who find it dehumanizing and Orwellian. Most alarming to the critics is that the system's separation of instruction from grading threatens the traditional, and to some, sacrosanct, relationship between teacher and student.

Man and Machine

The man behind this divisive system is Fred O. Kemp, an associate professor of English and designer of Topic (Texas Tech Online-Print Integrated Curriculum), the Web-based computer application on which Texas Tech's first-year composition program, or ICON (for Interactive Composition Online), is based.

Mr. Kemp emphasizes that the system was designed to solve a set of problems particular to a large research institution like Texas Tech Ð'-- namely how to use inexperienced graduate students to teach composition to 3,000 freshmen. Half of the graduate students entering Texas Tech have never taught before, says Mr. Kemp, or indeed even taken a basic composition course, having tested out of it in college. And with a 25- to 30-percent turnover rate, most who enter the system leave after three or four semesters.

"We have some folks who come in very talented, who like to teach and have a knack for it," says Susan M. Lang, an associate professor of English and a director of the composition program. "If you're a freshman and you luck out with the top 15 percent [of instructors], it's a phenomenal experience. But if you're in the bottom 15-20 percent, it's horrible."

Before ICON, says Mr. Kemp, the system for teaching freshman composition was rife with inconsistency. Or rather there was no system. Instructors drawn from creative writing, technical communication, rhetoric, and literature could not agree on either the content or criteria of good writing. Some instructors had students writing haiku and short stories, while others assigned lengthy research papers. At the beginning of each semester, says Mr. Kemp, the department dealt with wholesale movement between sections, while his office turned into a "complaint desk" for students carping about the program's inequities.

In the fall of 2001, when a graduate-student instructor was removed from the classroom for incompetence, Mr. Kemp, then director of the composition program, took over her two sections. Overburdened, he decided to divide the work of grading the 50 students' papers.

He tinkered with Texas Tech's homegrown database-driven software, Topic Ð'-- which, like the commercial courseware WebCT and Blackboard, allows students to file and store papers online Ð'-- so that other faculty members could read and grade the essays. The experiment was so successful that the new system was adopted programwide in 2002.

What is most radical about the system is the way it divides the labor of the traditional teacher into that of "classroom instructor" and "document instructor" or, in the local parlance, CI and DI.

Students meet once a week in a classroom with their classroom instructor to go over the finer points of grammar, style, and argumentation, and to discuss their weekly assignments, which are standardized across all 70-odd sections of the two required first-year composition courses. Each assignment cycle includes three drafts of an essay, reflective "writing reviews" commenting on students' own work, and two peer reviews of other students' work, all of which are submitted and stored online.

Document instructors, some of whom also work as classroom instructors, do the grading. Every piece of writing students produce is read by at least two anonymous graders from a pool of 60 to 70 graduate part-time instructors. The first reader reads, comments, and assigns a grade, from one to 100, to the document. When the second reader opens the file, he or she sees the essay, and the first DI's commentary, but not the grade. The second grader assigns a grade, and the computer averages the two. If the spread between the two grades is greater than eight points, the document goes to a third grader, and the two closest scores determine the composite grade. A student may appeal an assignment grade to his or her classroom instructor, who may choose to override it.

Mr. Kemp acknowledges his approach is controversial. The first semester the system was adopted, graduate students circulated a newsletter titled TOPIC Sucks, griping about the system's frequent technical glitches and industrial