Toegepaste Taalwetenschap in Artikelen - Volume 40, Issue 1, 1991

The results of two studies on writing processes are discussed. One study is about the relationship between cognitive activities and text quality in L1, the other on the relationship between Ll-writing profiency and cognitive activities during L2-writing tasks. It is argued that writing tasks in L2 should be more than mere formulating tasks and that both in L1 and in L2 teaching writing curricula should be organized around cognitive and meta-cognitive activities as generating, organizing, monitoring and evaluating in order to, reach intralingual and interlingual transfer of those activities.

This paper reports on the results of a detailed analysis of a functional writing assignment taken from the International Study of Written Composition of the International Association for the Evaluation of Educational Achievement (IEA). 15\16 year olds were asked to write a short notice of absence to their school principal. The corpus consisted of 299 texts.For the analysis of these texts four content elements deemed necessary for successful communication were chosen. These elements were:a = a reminder of (the time of) the appointmentb= informing of its cancellationc = giving the cause for the cancellation (i.e. illness)d = apologizing for the inconvenience.The data revealed four ways of organizing this type of notice. Each version is characterized by the first element mentioned, i.e. element a, b, c, or d. The versions differed in three ways: the ordering of the essential content elements, the number of incomplete texts per type, and the inclusion of additional information.Each of these three factors influenced the quality ratings of the texts. A formula weighing the various orderings and the absence of one or more of the information elements allowed a reliable reconstruction of these ratings. Whether the weights accurately estimate each of the features contribution to the functional adequacy of the communicative intent of the text will be the topic of a follow-up study.The data show that neither writing proficiency nor gender could explain the student's choice of one of the four versions. The fact that the versions were rated differently, raises doubts concerning the fairness of the task.

What is it that makes a text a good text? The answer to this question is of vital concern to education in writing. Evaluation of children's texts requires explicit and well-founded criteria for text quality. In this article it is argued that text structure should be considered as an important criterion for the evaluation of expository texts. Therefore, a model for the analysis of text structure is proposed. This model is based on and applied to explanations written spontaneously by 11 and 12 year old children, who were asked to explain something (e.g.: "who or what is Santa Claus?") to someone who does not know anything about the subject.The analyis results in a hierarchichal text representation in which the links between text segments (clauses) are made explicit. This is done in terms of the hierarchical position in the text (coordinating or subordinating links) and in terms of the coherence relation that holds between the clauses (e.g. Cause-Consequence, Claim-Argument). To provide problems of subjective interpretation it is proposed to make the analysis explicit by means of an algorithmic procedure that is to be developed . The heart of this analytic procedure is that there are two grounds to connect text segments: referential continuity and a line of events.On the basis of the analysis three types of text structure problems can be identified in children's texts: Discontinuity, incompleteness and ambiguity. All three can also be revised on the basis of the analysis. In the case of discontinuity, the information must be reorganized, in the case of incompleteness information must be added and in the case of ambiguity the relation between segments must be made explicit.The analysis of text structure provides the basis for a criterion for text quality assessment. After all, the major goal of a writer trying to explain something to a reader is that the reader understands what the writer means. Understanding a text means constructing a coherent representation of that text. If readers have trouble in constructing such a representation, the text should be evaluated negatively. This is exactly what the analysis predicts: It is difficult to construct a coherent representation at the points in the text structure where discontinuity, incompleteness and ambiguity occur.

Test results should be determined as little as possible by chance factors. Such factors harm the reliability of results. In testing writing proficiency open-ended tasks may involve a variety of chance factors affecting assessment a.o. It is well-known that when a number of teachers rate the same written work, the outcomes tend to differ considerably from each other. The subjectivity showing up here also affects the assessment of modern foreign language writing proficiency in school-based exams. The use of different measures may be hard to rule out, but is unacceptable as a matter of principle. For this reason CITO has been making attempts to improve the agreement among raters of foreign language writing proficiency. Research has been done to find out if information offered to teachers concerning the severity, or otherwise, of their assessments enhanced inter-rater agreement. The subject of this investigation was based on the following argument: teachers are often unaware of how their assessments compare to those of colleagues. If there were a means enabling them to determine (independently and anonymously) whether they fall into the category of mild or severe assessors, this information might cause them to change their 'deviant' behaviour and aim at the mean. If many teachers were to calibrate their evaluation in this way, the objectivity of assessment might be improved.In order to find out if such an effect occurs at all, a test was carried out in which 26 teachers rated 25 letters written by pupils (in German, French or English). They were asked to follow precise instructions. The results were reported back to them. After over a year had passed, the teachers were asked to rate the same letters once again following the same instructions as before. They were reminded of the results of the previous round (by such comments as: 'last time your assessments were pretty severe'). The inter-rater agreement turned out to be neither higher nor lower than the first time. The information about relative mildness or otherwise apparently had not had the desired effect. One explanation could be that, contrary to what had been hoped, teachers are not really able to adjust their rating. The fact that the results of raters in the 'average' group (who, of course, were not supposed to adjust their rating) showed quite some variation over the two rounds (the mean intra-rater agreement was .79) would seem to support this explanation.

In the assessment of writing ability it is difficult and costly to rate the students' texts reliably. Using more structured writing tasks and rating procedures could help avoiding some of the assessment problems. However, structured tasks and rating procedures threaten the validity of the writing assessment. This balance between (rating) reliability on the one hand and validity on the other hand is the main focus of this paper.In the present study five different types of writing task were evaluated. The writing tasks represented different degrees of structuring. Two tasks could be regarded as direct measurements, two as semi-direct measurements and one as an indirect measurement.A way to improve rating reliability is to structure the rating procedure as opposed to structuring the writing task. Two scoring methods were compared with respect to the ratings they produced for two tasks: the specified and the stuctured task. The methods consisted of (1) essay scales with example essays as 'range finders', and (2) scoring according to strict scoring guides (counts). The structuring of both the writing task and the rating procedure were expected to simplify the assessment of writing ability and to improve the reliability of the measurements.The effects of writing task and rating procedure on reliability of individual raters (five raters with the direct measurements, three with the semi-direct measurements) were analysed with ANOVA's. The results did not completely support the theoretical assumptions. However, as expected, the semi-direct tasks were rated far more reliably than the direct tasks. The (small) differences between the two direct respectively two semi-direct tasks showed a positive 'structuring effect', i.e. gain in rating reliability. Furthermore, the rating 'with scoring guides' was shown to be more reliable than the rating with essay scales. This difference was related to the aspect of the texts being rated: rating reliability appeared to vary between the rating procedures when content and organization were being rated, whereas no such difference was found in rating language usage.Possible instrument bias was studied as a way of evaluating the validity of the writing assignments. No bias with respect to sex could be demonstrated, but there was a bias with respect to writing ability. Proficient writers seemed to be unable to produce a distinctive performance in semi-direct and indirect tasks, whereas less proficient writers seemed especially to benefit from the 'simplicity' of these more restrictive tasks. These findings appeared most strongly in relation to the content and organization scores.In line with other evaluations of the validity of writing assignments (cf. Schoonen, 1991), one must conclude that the gain in reliability as a result of the structuring of the writing assignment also means a loss in validity, especially with respect to content and organizational aspects of texts. The rating of the essays is best done using essay scales rather than strict scoring guides (counts), especially when it concerns language use.

In a study of composition skills of students in the highest grades of primary school four experimental tasks were administered to 32 subjects. Each was designed with the purpose to find specific knowledge and strategies for producing written text that are available to children of this age. In all tasks the children were asked to concentrate exclusively on certain difficulties of text production, to wit: a) the selection of global ideas, b) the organisation of ideas, c) the translation of ideas into written sentences, d) reviewing and revising text.A theoretical rationale for the task construction is given and one of the tasks (a) is explained more fully. Results of this task, based on a classification of arguments the students gave for selecting ideas are reported and some conclusions are given focussing on the implications of the study for writing education.

In our article we describe the functions of so-called 'text plans1 in two different processes of text production: (1) dictating business letters and (2) writing governmental policy issue papers. Text plans are defined as: abstract, schematic representations of the text a writer/dictator wants to convey.We look upon dictating business letters and writing policy issue papers as two writing tasks. In our article we explore the differences between the two tasks in terms of the constraints the dictator and the policy writer should meet. Our research makes clear that dictators strive for first-time-final versions and that planning the text is a necessary requirement for effective dictating. While dictating, the dictator follows the exact line of his plan. Writing policy issue papers, however, is mainly a collaborative writing task. Policy issue papers are written in groups which strive for internal consensus about the issues (and the text) at stake. Text plans are uses by policy writers to divide tasks, to guarantee cohesion between the different contributions and as the first negotiation object.

From various studies there is evidence that revision tasks are helpful in the development of writing skills. In order to know more about the mental activities student-writers undertake while revising their own texts, we have made an analysis of 90 writing assignments written and revised by pupils of 12 years old. In an attempt to help pupils in the revision phase, we added 2 kinds of feedback at the sentence level:- minimal feedback which merely indicates the sentence problem;- diagnostical feedback which indicates a solution for the sentence problem.The research concentrated on the question which kind of feedback had resulted in adequate revisions at the sentence level. Analysis showed that- each kind of feedback was given equally frequent, regardless of the nature of sentence errors;- opaque sentence errors were revised successfully more often than sophisticated sentence errors ( ρ < .01);- minimal feedback led to more successful revisions than diagnostical feedback ( ρ < .01).These results suggest that student-writers are not able to make use of diagnostical comments and that further research should be done in order to discover which kind of feedback is the most succesful at sentence level.

Eight- to ten-year-olds no longer write the way they talk: they make a distinction between speech and writing. However, they lack specialized knowledge on written language. A key issue is their lack of knowledge on sentences. The fully-fledged sentences of written language do not come naturally to language users. If children demarcate sentences in writing, they are guided by their tacit grammatical knowledge only. This knowledge misguides them.Sentences can be considered as writers* building blocks, the smallest units with which to plan and to organize text.Results of an experimental study into the effects of grammar instruction upon writing by young children not only suggest that it is very well possible to teach junior writers what are sentences in the a-rhetorical context of grammar class; children spontaneously put to use this explicit, declarative knowledge while writing, too. Thus, it seems, knowledge transfers spontaneously.Moreover, effects in a discrete point test may indicate that instruction in the grammar of sentences has students start exploiting sentences for what they are: a scaffold for writing.

In this article an attempt will be made to explain the nature of the non - native character of Italian L2 texts written by advanced Dutch university students by analyzing the various ways in which the nominal group is formed in these texts. It will be argued that the non -native character of the L2 nominal group is to be ascribed to an imbalance in the use of the possibilities offered by the target language, an imbalance that is the outcome of Ll-based choices.

In the period 1985-1990, software and written learning materials have been developed in the project Computer-assisted Writing Instruction, a research-and-development project of the University of Twente. The courseware was intended to serve as a prototype for writing instruction in which much attention is to be paid to text planning, before and during writing. In the experimental lessons students used a word processor and a planning support program as an aid for composing their texts.An educational innovation, such as the use of the computer as a tool or approaching writing instruction in a process-oriented way, displays its effects on all levels in the instructional situation. One intends to influence the writing achievements of students positively, but at the same time the daily routine in classes changes, both for teacher and for students. Various effects of the experimental materials and the lessons realized with it, were studied in nine third grade classes of MAVO and HAVO (lower and higher general secondary education) and VWO (pre-university secondary education). A selection of results with regard to the effects on the writing achievements of students and with regard to changes in their writing processes are being presented.From the study it appeared that students improved their writing achievements during the experimental course: they wrote better letters after the course than they did before. The writing processes of students who wrote a letter (supported by the computer program) also appeared to be changed. The required letter was planned at a higher text level after the course. However, for some of the participating students this change appeared to be immature.

The proposed research and application programme can be seen as a text-constitutive programme, which enables the user to appeal to computerized information, while planning, formulating and revising a particular writing task.The computerized information is stored in interactive modules, i.e. 1) a CONST-specific thesaurus and an adapted version of the electronic Van DALE 12 dictionary, 2) an algorithmic framework, actualized by a number of question-packages regarding business texts and memos, and 3) a set of corresponding predefined and ad hoc constructed structural descriptions.Important topics of research are the symmetric internal and external electronic communication links inside and between the various modules, as for example the internal link between the two thesauri or the external links between structural description, question-package and thesauri.What can be demonstrated, is a prototypical framework, in which each CONST-component is represented properly and in which the various communication links have been actualized.