Assessing Text-Based Writing of Low-Skilled College Students

Published on Mar 1, 2018
· DOI :10.1007/s40593-016-0122-z
Dolores Perin17
Estimated H-index: 17
(Columbia University),
Mark Lauterbach2
Estimated H-index: 2
(CUNY: City University of New York)
The problem of poor writing skills at the postsecondary level is a large and troubling one. This study investigated the writing skills of low-skilled adults attending college developmental education courses by determining whether variables from an automated scoring system were predictive of human scores on writing quality rubrics. The human-scored measures were a holistic quality rating for a persuasive essay and an analytic quality score for a written summary. Both writing samples were based on text on psychology and sociology topics related to content taught at the introductory undergraduate level. The study is a modified replication of McNamara et al. (Written Communication, 27(1), 57–86 2010), who identified several Coh-Metrix variables from five linguistic classes that reliably predicted group membership (high versus low proficiency) using human quality scores on persuasive essays written by average-achieving college students. When discriminant analyses and ANOVAs failed to replicate the McNamara et al. (Written Communication, 27(1), 57–86 2010) findings, the current study proceeded to analyze all of the variables in the five Coh-Metrix classes. This larger analysis identified 10 variables that predicted human-scored writing proficiency. Essay and summary scores were predicted by different automated variables. Implications for instruction and future use of automated scoring to understand the writing of low-skilled adults are discussed.
  • References (72)
  • Citations (6)
📖 Papers frequently viewed together
108 Citations
14 Citations
1 Author (Stacey Murrell)
78% of Scinapse members use related papers. After signing in, all features are FREE.
#1Neethu Robinson (BITS: Birla Institute of Technology and Science)H-Index: 6
#2Ali Danish Zaidi (University of Tübingen)H-Index: 2
Last. Ranganatha Sitaram (UC: Pontifical Catholic University of Chile)H-Index: 25
view all 7 authors...
Recently, studies have reported the use of Near Infrared Spectroscopy (NIRS) for developing Brain–Computer Interface (BCI) by applying online pattern classification of brain states from subject-specific fNIRS signals. The purpose of the present study was to develop and test a real-time method for subject-specific and subject-independent classification of multi-channel fNIRS signals using support-vector machines (SVM), so as to determine its feasibility as an online neurofeedback system. Towards ...
16 CitationsSource
#1Scott A. Crossley (GSU: Georgia State University)H-Index: 29
#2Kristopher Kyle (GSU: Georgia State University)H-Index: 11
Last. Danielle S. McNamara (ASU: Arizona State University)H-Index: 50
view all 3 authors...
Abstract An important topic in writing research has been the use of cohesive features. Much of this research has focused on local and text cohesion. The few studies that have studied global cohesion have been restricted to first language writing. This study investigates the development of local, global, and text cohesion in the writing of 57 s language (L2) university students and examines the effects of these cohesion types on judgments of L2 writing quality Growth is observed in the use of a n...
18 CitationsSource
#1Steve Graham (ASU: Arizona State University)H-Index: 53
#2Michael Hebert (NU: University of Nebraska–Lincoln)H-Index: 16
Last. Karen R. Harris (ASU: Arizona State University)H-Index: 59
view all 4 authors...
This study examined the number of writing samples needed to obtain a reliable estimate of young struggling writers’ capabilities. It further assessed if performance in one genre was reflective of performance in other genres for these children. Second- and third-grade students (81 boys, 56 girls), who were identified as struggling writers in need of special assistance by their teacher and scored at the 25th percentile or lower on a norm-referenced story-writing test, wrote four compositions: a st...
18 CitationsSource
#1Charles A. MacArthur (UD: University of Delaware)H-Index: 36
#2Zoi A. Philippakos (UNCC: University of North Carolina at Charlotte)H-Index: 3
Last. Stephen Graham (ASU: Arizona State University)H-Index: 51
view all 3 authors...
The purpose of the current study was to develop and validate a measure of motivation for use with basic college writers that would measure self-efficacy, achievement goals, beliefs, and affect. As part of a design research project on curriculum for community college developmental writing classes, 133 students in 11 classes completed the motivation scales at the beginning and end of the semester along with measures of writing quality. Single factors were found for self-efficacy and affect. For go...
13 CitationsSource
Composition faculty find themselves ancillary to the political reform movements promoting assessment within their own institutions. They blink an eye, and suddenly some victor is nailing a manifesto right on their classroom door, directly affecting their programs and the lives of their students. The only recourse seems frustrated acceptance or angry protest. But . . . [w]e discovered, perhaps mostly by luck, that the real site of writing assessment is not so much a battle zone or a contested eco...
8 Citations
#1Asad Abdi (Information Technology University)H-Index: 7
#2Nik Rumzi Nik Idris (Information Technology University)H-Index: 20
Last. Ramiz M. Aliguliyev (ANAS: Azerbaijan National Academy of Sciences)H-Index: 18
view all 4 authors...
Background Summarization is a process to select important information from a source text. Summarizing strategies are the core cognitive processes in summarization activity. Since summarization can be important as a tool to improve comprehension, it has attracted interest of teachers for teaching summary writing through direct instruction. To do this, they need to review and assess the students' summaries and these tasks are very time-consuming. Thus, a computer-assisted assessment can be used to...
6 CitationsSource
#1Barbara Carretti (UNIPD: University of Padua)H-Index: 26
#2Eleonora Motta (UNIPD: University of Padua)H-Index: 1
Last. Anna Maria Re (UNIPD: University of Padua)H-Index: 13
view all 3 authors...
Several studies have highlighted that children with reading comprehension difficulties also have problems in tasks that involve telling a story, in writing or verbally. The main differences identified regard poor comprehenders’ lower level of coherence in their productions by comparison with good comprehenders. Only one study has compared poor and good comprehenders’ performance in both modalities (oral and written), however, to see whether these modalities differently influence poor comprehende...
11 CitationsSource
#1Timothy Shanahan (UIC: University of Illinois at Chicago)H-Index: 30
19 Citations
In this experimental study, 151 middle school students explored 3 historical controversies, first reading and discussing primary source documents in groups, then writing arguments on their own. Students were either randomly assigned to an experimental condition, using argumentative schemes and critical questions as guides during discussions, or to a comparison condition in which a traditional set of questions was used to guide discussions. Students in both conditions read the same historical con...
15 CitationsSource
#1Joshua Wilson (UD: University of Delaware)H-Index: 8
#2Natalie G. Olinghouse (UConn: University of Connecticut)H-Index: 13
Last. Gilbert N. Andrada (Connecticut State Department of Education)H-Index: 3
view all 5 authors...
Abstract Students who fail state writing tests may be subject to a number of negative consequences. Identifying students who are at risk of failure affords educators time to intervene and prevent such outcomes. Yet, little research has examined the classification accuracy of predictors used to identify at-risk students in the upper-elementary and middle-school grades. Hence, the current study compared multiple scoring methods with regards to their accuracy for identifying students at risk of fai...
11 CitationsSource
Cited By6
#1Olaf Zawacki-Richter (University of Oldenburg)H-Index: 12
#2Victoria Irene Marín Juarros (University of Oldenburg)H-Index: 7
Last. Franziska Gouverneur (University of Oldenburg)H-Index: 1
view all 4 authors...
According to various international reports, Artificial Intelligence in Education (AIEd) is one of the currently emerging fields in educational technology. Whilst it has been around for about 30 years, it is still unclear for educators how to make pedagogical advantage of it on a broader scale, and how it can actually impact meaningfully on teaching and learning in higher education. This paper seeks to provide an overview of research on AI applications in higher education through a systematic rev...
4 CitationsSource
#1Edward Lehner (CUNY: City University of New York)H-Index: 4
#2John R. Ziegler (CUNY: City University of New York)H-Index: 1
#1Charles A. MacArthur (UD: University of Delaware)H-Index: 36
#2Amanda Brooke Jennings (UD: University of Delaware)H-Index: 1
Last. Zoi A. Philippakos (UNCC: University of North Carolina at Charlotte)H-Index: 3
view all 3 authors...
The study developed a model of linguistic constructs to predict writing quality for college basic writers and analyzed how those constructs changed following instruction. Analysis used a corpus of argumentative essays from a quasi-experimental, instructional study with 252 students (MacArthur, Philippakos, & Ianetta, 2015) that found large effects (ES = 1.22) on quality of argumentative writing. Coh-Metrix (McNamara, Graesser, McCarthy, & Cai, 2014) was used to analyze the essays for lexical and...
1 CitationsSource
#1Haiying Li (RU: Rutgers University)H-Index: 7
#2Zhiqiang Cai (U of M: University of Memphis)H-Index: 30
Last. Arthur C. Graesser (U of M: University of Memphis)H-Index: 77
view all 3 authors...
In this study we developed and evaluated a crowdsourcing-based latent semantic analysis (LSA) approach to computerized summary scoring (CSS). LSA is a frequently used mathematical component in CSS, where LSA similarity represents the extent to which the to-be-graded target summary is similar to a model summary or a set of exemplar summaries. Researchers have proposed different formulations of the model summary in previous studies, such as pregraded summaries, expert-generated summaries, or sourc...
4 CitationsSource