Volume 1, Number 1

Teachers respond to Halio

This article first appeared as "Computer Teachers Respond to Halio" in Computers and Composition 7 (1990): 73-79. It is posted here by permission of principle author, John Slatin. It is preceded by a brief synopsis, which contains links to sites within the article.

Synopsis: <1> In "Computer Teachers Respond to Halio" teachers of computer-assisted writing classes note that Marcia Peoples Halio's article is methodologically flawed, but that the currency that it has been given by administrators prompts a careful critique. They first note that Halio makes no direct comparisons between essays produced using Macintosh computers and those written on DOS based machines. Instead Halio presents the results of the Writers Workbench text analysis program, which quantifies information about a text, but says nothing about its quality or content. In addition, the results of the workbench analysis, according to the computer teachers, remain open to interpretation. The teachers also question the population of Halio's study and note the failure of Halio to consider background, ethnicity, gender and other relevant contextual information in the study. The teachers also note Halio's failure to provide information about the training given to students using the different platforms. Finally, the writers conclude that computers are changing composition and suggest that the questions raised by Halio's article are valid, but the premature conclusions that she draws from them are dangerous.

Return to Not Maimed but Malted or to the critique of Halio and fonts.

Article:

Computer Teachers Respond to Halio

<2> In "Student Writing: Can the Machine Maim the Message?" (Academic Computing 4 [January 1990] ), Marcia Peoples Halio of the University of Delaware reports that first-year writing students using Macintosh computers produced consistently poorer essays than their peers working with MS-DOS computers. She attributes the difference in the quality of student writing to differences between the computers themselves--specifically to the difference between the Macintosh's graphical interface and the character-based, command-line interface of the MS-DOS machines. Her own experience, along with that of four instructors who taught on both Macintoshes and IBMs during the Fall 1988 semester, "seems to demonstrate," she says, that "using the Mac or the IBM could have [a significant] effect on students' writing"(17).

<3> As we shall indicate below, Halio's article is so seriously flawed by methodological and interpretive errors that it would probably have been dismissed had it appeared in a journal directed to an audience of professional writing teachers. Publication in Academic Computing has given it wide circulation, however, not only among faculty members involved with writing instruction, but also among administrators responsible for purchasing equipment for their campuses. Its potential impact is therefore considerable.

<4> This letter grows out of discussions taking place over a BITNET discussion loop called Megabyte University (moderated by Fred Kemp at Texas Tech University) between January 30 and March 2, 1990. Megabyte University "enrolls" some 70 people interested in writing instruction, including faculty members and graduate students from universities and colleges across the United States. Approximately a dozen members became involved in the discussion of Halio's article. Most were initially inclined to dismiss the article as trivial, until faculty members participating both in Megabyte University and in another loop called HUMANIST, with some 600 members in 20 countries (edited by Willard McCarty at the University of Toronto) reported receiving photocopies of Halio's article from deans and other administrators, with comments to the effect that Halio has "proved" the inferiority of the Macintosh as a machine for writing instruction.

<5> These reports have persuaded several individuals that we should explain the problems we see in Halio's article to readers of Academic Computing who may not be aware of control procedures used in writing research and who may misinterpret the numerical data Halio provides. The signers negotiated the composition of this "corporate" reply in open discussion on Megabyte University, posting drafts and incorporating the ensuing comments. The result has been shaped not only by those whose names appear below, but also by the objections and counterarguments of individuals who have chosen not to sign; the latter are not, of course, accountable for the contents of this letter. We do not seek to demonstrate that Halio is wrong. Our point is that the University of Delaware experience may not lead to the conclusions Halio reaches, and that a far more careful study is required.

<6> Halio offers the following evidence:

1. Her own qualitative observations about the writing of students taught by herself and by several other instructors.
2. Results obtained from the Writer's Workbench text analysis program's Style module which she believes to have "confirmed" her observations (18).
3. The remarks of four unidentified instructors who responded to a query from her.

<7> Halio also quotes at length from three student papers produced on the Macintosh (17). They are indeed poorly written, though they might have been much worse without deserving the thrashing Halio gives them (17). Furthermore, since much of Halio's contempt is directed at the topics chosen by Macintosh users(17), how did two of these students came up with identical topics ("American Eaters") if, as Halio claims, they were given only general "writing suggestions" (17)?

<8> [return to top] Direct comparison of these examples with sample essays produced on IBM computers would have been extremely helpful, but Halio offers no samples of writing done on an IBM. Instead, she uses results obtained from Writer's Workbench to make her case against the Macintosh. Macintosh users wrote "fewer complex sentences" than IBM users, used more "to be" verbs, and wrote shorter sentences; their essays also received lower scores on the Kincaid readability scale- - 7.95 for Mac users, as opposed to 12.1 for students writing on the IBM. These results, says Halio, "confirmed [her] initial impressions" (18); but they do not necessarily mean what she says they do.

<9> The Writer's Workbench Style program supplies a great deal of information about a piece of text, but none of it concerns the content or the quality of that text. That is, the program cannot tell us anything that an essay says, or about its value; and it is likewise unable to determine the relationship between content and the stylistic features it is designed to measure.

<10> Moreover, its analyses of those features have only about eighty percent accuracy, and the program's output can confuse people who don't know how to interpret it. Three of Halio's Writer's Workbench measurements are suspect even if the program's information is accurate:

(a) Readability tests measure readability, not writing ability. In general, a high readability score means that the writing is hard to understand. Clear business writing and general magazine writing, for instance, normally score from 8 to 10 on the Kincaid scale, not 12.1.
(b) Sentence length, as Halio says, is related to readability. In general, longer sentences are harder to understand. Most writers of non-technical prose average only 16-20 words per sentence, and many style analysis programs (and many readers) would reject the 22.6-word average that Halio cites as desirable.
(c) "To be" verbs are too high in both samples that Halio cites. Even the 23 percent scored by IBM users in Halio's sample exceeds the 15 to 19 percent usually found in professional writing.

<11> Even the formalistic measures Halio uses are open to multiple interpretations, then, and without sample texts for comparison, they do not by any means prove that students writing on the IBM produced significantly better work than those who used the Macintosh. Certainly these data offer no grounds for concluding that the computers caused the differences Halio perceived in the students' writing.

<12> [return to top] Halio provides a similarly misleading description of the Delaware student population. She says that "all students in the computer sections [of Delaware's first-year writing course] have roughly comparable levels of writing ability" "because their SAT scores as well as the results of a placement essay have put them in the medium writing- ability range (they did not qualify for the Honors Program, nor were they placed in the remedial sections)" (17). This "medium writing-ability range" is wider than Halio implies: one presumes, for instance, that instructors felt justified in using the full grading scale--A to F-- to delineate differences among individual students.

<13> A more useful study would provide detail about the students, their backgrounds, and the attitudes toward writing they brought with them into the classroom-- in other words, information about the factors influencing the students' classroom performance and, indeed, their initial choice of which sections to take. Halio completely ignores information crucial to evaluating student writing-- information about the students' racial, ethnic, and class affiliations, about their gender, and (not least, in this context) about their previous experience with computers.

<14> Halio says that students were free to choose between Macintosh- and IBM-based sections of the first-year writing course; and it's precisely because the students were free to choose that we need to know so much more about them and the reasons for their choices. For example, did students with little experience in writing on computers choose the "friendly" Macintosh?

<15> Finally, a reliable study would provide specific information about Delaware's writing curriculum, and the ways in which it integrates computers into the composing process and the curriculum generally. Halio says nothing about the curriculum, however. Moreover, she implies that students are left to figure out for themselves how best to use the computers for writing: "after approximately one and one-half hours of instruction," she writes, "they then work to improve their computer skills from self-paced handouts prepared by our Academic Computing Support services. They use the public sites on their own time to write their papers" (17; emphasis added).

<16> Ninety minutes may be enough training for students to learn the basic elements of word processing, though even that depends in large part on their familiarity with word processing and the particular word processing package involved, which Halio does not identify. (Both the University of Texas at Austin and Iowa State University, for instance, offer Beginning, Intermediate, and Advanced courses, each an hour and a half long, on both MS-DOS and Macintosh versions of Microsoft Word.) But there is a considerable difference between using a word processor to enter and manipulate text which has already been composed, and using the word processor as a fundamental part of the composing process itself. Even trained, professional writers making the transition from pen and legal pad or from typewriter to word processor require considerably longer than an hour and half's training to integrate the word processor completely into their methods of composition.

<17> [return to top] For all the flaws in her article, Halio has raised serious questions about the effects of hardware and software design upon those who write with computers, and we must certainly investigate those effects more fully. As we do so, we will have to consider the strong possibility that we may need to adapt our writing curricula to computer-based (and graphics-oriented) writing technology. The computer changes writing practices, and the further the technology diverges from traditional practice, the more teaching practice has to take that shift into account. It isn't in raising the questions, but in claiming to have answered them and in rushing prematurely to publication rather than waiting for the results of the "more carefully controlled experiment" she says she is now conducting (45), that Halio--and Academic Computing--have acted irresponsibly.

John M. Slatin
Director, Computer Research Lab
Associate Director, Lower-Division English
University of Texas at Austin
EIEB360@UTXVM.BITNET

Trent Batson
Director, Project ENFI
Gallaudet University

Robert Boston
Department of English
Iowa State University

Michael E. Cohen
User Relations Liaison
Humanities Computing Facility
UCLA

Louie L. Crew
Department of English
Rutgers University, Newark

Lisa Gerrard
Rhetoric Program
UCLA

Gail Hawisher
Department of English
Purdue University

Edward M. Jennings
Department of English
SUNY Albany

Michael Joyce
Center for Narrative and Technology
Jackson (Michigan) Community College

Nancy Kaplan
Director, Writing Workshop
Cornell University

Stuart Moulthrop
Expository Writing Program
Yale University

Cynthia L. Selfe
Humanities Division
Michigan Technological University

John Slatin
Director, Computer Writing & Research Lab
Division of Rhetoric and Composition and
Department of English
University of Texas at Austin
Austin, TX 78712
(512)471-8743/fax (512)471-4353
jslatin@mail.utexas.edu

Move on to the conclusion, return to Not Maimed but Malted, or return to the critique of Halio and fonts.
Return to Daniel's home page or to the CWRL home page.
iamdan

CWRL Table of Contents