**2.2 Materials**

Two essay topics were selected in accordance with the textbook used. One topic was creating an original robot, while the other was explaining an interesting job (see Appendix 1). Expected readers (i.e., classmates and teachers) of the essay were specified in the writing prompts. On the assignment paper, the following instructions were given concerning essay writing: First, write an essay of approximately 400 words. Second, pay attention to the structure of the essay, which is supposed to have an introduction, body paragraphs, and a conclusion. Third, make sure that each paragraph has a topic sentence and several supporting sentences, whereas a concluding sentence is optional. Fourth, provide adequate details and appropriate examples, by adjusting the concrete or abstract information. Fifth, cite at least one source and use citations appropriately. Do not plagiarize. Under the instructions, the assessment criteria were also written, which included a) content and source use: 10 points, b) organization and logic: 10 points, c) language use: 5 points, and d) reader awareness: 5 points.

A self-reflection worksheet regarding writing the first draft was prepared, where the writer was asked to write about how they intended to convey their ideas (see Appendix 2). Self-reflection was set for heuristic purposes [16]. More precisely, the writer had the opportunity to reflect on their writing process and product critically. In addition, the participants were instructed to evaluate the strengths of their draft and points to be improved considering the following aspects: content, source use, organization and logic, language use, and reader awareness. These aspects corresponded to the criteria used to mark essays, which were shared with the participants. The specific aspects in the self-evaluation sheet were intended to help the writer critically analyze and identify what is good or not in each aspect of the draft. It is regarded as a preparation stage before conducting peer feedback because writers need to be critical readers of peers' essays.

The written peer feedback sheet was structured in a problem-solving format (see Appendix 3). More specifically, readers were encouraged to fill in the section of "problems and their reasons" and "suggestions for improvement" regarding content, the overall structure, paragraph structure, logic, and source use. If readers found good points about each aspect of the essay, they were advised to provide positive comments. In giving feedback comments, it is important to provide both praise and comments regarding improvement because the former often strengthens the writer's confidence and the latter helps the writer revise and improve the draft [14, 21]. In response to the peer feedback, the peer feedback sheet also had a section

*Approach to Writing from Readers' Perspectives: Incorporating Self-Evaluation, Peer Feedback… DOI: http://dx.doi.org/10.5772/intechopen.100308*

named "self-reflection after revisions," in which the writer reflects on the revision process. The writer was supposed to check which peer feedback comment is incorporated in revising the essay. More specifically, the writer was instructed to judge whether the suggestion was "fully accepted," "partly accepted," or "not accepted" and provide the reason.

In the self-reflection questionnaire, the writer answered the following questions. (1) How did you revise the draft and in what point does the final draft differ from the first draft? (2) Was peer feedback useful in revising the draft? Select one of the four: very useful, useful, not useful, not useful at all. (3) Provide the reason for your choice as detailed as possible. The questionnaire was written in Japanese.

### **2.3 Procedures**

The participants learned a writing process in class, where brainstorming, making an outline, writing a draft, revising, and editing were explained. They had already learned the essay structure and citation, had written a paragraph and an essay, and had experienced peer feedback when paragraph writing was assigned. Following the instructions, the participants first brainstormed ideas using a mind mapping approach for eight minutes and then talked about them in pairs asking questions or explaining ideas. After that, they were advised to decide what they wanted to write about based on the ideas on the mind map and make an outline from it. The participants were instructed to write a draft of a 400-word essay as homework, reflect on the writing process, and evaluate it by providing strengths and points to be improved in the draft. After writing the draft and self-evaluation, both were submitted to an online system.

Twenty-minute peer feedback sessions took place in a computer-assisted language learning classroom, where the English class was usually held, and thus, the participants were familiar with the classroom environment. They were instructed to read peers' essays in pairs and make comments on a digital peer feedback sheet using their L1 (i.e., Japanese). This is because the use of L1 enables writers to express their thoughts and comments fluently. Although peer feedback can be given in oral and/or written forms, a written form was used in this study so that the researcher could follow the participants' comments and progress in revision [16]. Compared to oral feedback, the written feedback form allows writers and readers to consider, collaborate, and reflect on essays more actively [16]. Before the peer feedback sessions began, the following instructions were given to the participants: (1) explaining the purpose of peer feedback, (2) emphasizing the value of honest evaluation, (3) encouraging them to give praise and constructive comments, (4) encouraging them to give clear and specific comments, and (5) making sure that the writer, not the reader, can decide what and how to revise [14]. Additionally, examples of effective and ineffective feedback comments were shown to the participants. These examples were made available on the computer screen, so they could look at them anytime during peer feedback sessions. The participants were also informed that they could ask the teacher any questions during the peer feedback session.

Each participant took part in peer feedback sessions twice in pairs; therefore, they had opportunities to read and evaluate two essays written by peers. Providing students with more than one peer feedback session with different peers is important because it allowed them to receive more comments from different readers' viewpoints and also to read multiple drafts to learn variations of writing. Furthermore, even when one peer cannot make useful or appropriate comments, another peer may provide constructive feedback. Thus, two rounds of peer feedback sessions could guarantee a positive experience for both readers and writers to some extent. After the participants received feedback comments, they were advised to read them

thoroughly and revise their draft based on accordingly and what they had learned from the peer feedback sessions. At this point, it is noteworthy that the researcher emphasized that the writer has ownership over the essay and a right to decide what to revise and that they do not have to take all the advice given by peers [16]. Thus, the participants were encouraged to think carefully about how to revise and make a decision about it by themselves. After completing the final draft, they filled in a questionnaire, where they wrote about how they revised the draft and what they thought about the peer feedback activities. Their peer feedback sheets, which were filled with feedback comments, final drafts, and questionnaires were collected.

### **2.4 Scoring and data analysis**

The final drafts were scored based on the criteria created by the researcher for this study. It had four aspects. First, content and source use (10 points) serve to identify whether the essay corresponds to the essay question or theme, whether the essay is interesting to read, and whether sources are cited appropriately. Second, organization and logic (10 points) focus on the essay, paragraph structure, and logical flow of the essay. Third, language use (5 points) identifies whether an essay is written in an appropriate language. Finally, awareness of readers (5 points) identifies whether the writer considers readers' expectations and adjusts the content and expressions accordingly. A total of 122 essays were marked using the criteria described above by the researcher. SPSS Statistics 27 was used to perform a chisquare test and analyze the correlations between the essay scores and the views on the usefulness of peer feedback.

Considering the analysis of self-evaluation of the first drafts, the participants' responses were divided into strengths and weaknesses of their essays and categorized into five aspects: content, citation, organization and logic, language use, and reader awareness, which followed the criteria used in the rubric. The number of each category was calculated.

The self-reflection questionnaire regarding peer feedback was analyzed using the free software named KH Coder 3 [22]. This software is used for text mining or quantitative content analysis, which is an automated analytical method for extracting a large amount of textual data. It enables the user to analyze textual data objectively by quantifying frequent words and identifying and visualizing the relationships between different words. In this study, the written comments of the participants were analyzed using KH Coder 3. First, the words used frequently in the comments were quantified and listed to determine the aspects of the peer feedback the participants perceived positively or negatively. Second, the function of the co-occurrence network was employed to reveal the relationship between the different words pointed out by the participants. Third, correspondence analysis was conducted to determine whether essay scores corresponded to the evaluation of peer feedback. More specifically, we analyzed which groups of writers are associated with certain concepts of revision and peer feedback.

## **3. Results and discussion**

#### **3.1 Basic results of the essays**

The essays were divided into three groups (i.e., high, medium, and low) depending on the scores. **Table 1** shows the descriptive features of the three essay groups. The results of one-way independent analysis of variance (ANOVA) showed that these groups differed significantly in scores (*F* (2, 119) = 284.263, *p* < .001, *η* 2 = .83). *Approach to Writing from Readers' Perspectives: Incorporating Self-Evaluation, Peer Feedback… DOI: http://dx.doi.org/10.5772/intechopen.100308*


#### **Table 1.**

*Descriptive statistics of the essay scores.*

Tukey's post-hoc test indicated that each group significantly differed in mean scores (*p* < .05).

#### **3.2 Writers' self-evaluation of essays**

The results of the participants' self-evaluation of their first drafts showed that the number of positive comments related to strengths (*n* = 407) of the essays was nearly twice as many as those related to weaknesses (*n* = 220, see **Table 2**). This tendency was observed for every aspect of the evaluation criteria. This implies that writers tend to focus more on the strengths of their writing than on their weaknesses. This tendency was also observed in peer feedback comments previously conducted with Japanese university students [19].

Among the positive comments, the number of comments concerning "reader awareness" was the most frequent (*n* = 108), accounting for 26.54% of the positive comments. This finding suggests that most of the writers paid attention to readers' expectations of the essay and made an effort to accommodate their essays to their readers (i.e., classmates and a teacher). The second most frequent positive comments were related to "organization and logic" (*n* = 93, 22.85%), indicating that the writers carefully constructed the introduction, body, and conclusion or each paragraph. The participants made fewer comments on "citation" (*n* = 52, 12.78%) and "content" (*n* = 70, 17.20%), which implies that they were not necessarily confident in these areas of essays.

Considering the comments concerning the weaknesses of the essays, the most frequent aspect pointed out was "language use" (*n* = 67), which accounted for 30.45% of the total number of comments on weaknesses. This indicates that "language use" is the most concerning aspect for the participants. Furthermore, the total number of comments regarding "language use" was the most frequent (*n* = 151) among the five aspects. This result implies that L2 student writers are concerned about local issues of "language use," which supports the finding that the language-related style of writing was the most prominent aspect of students' feedback comments [18]. Furthermore, "citation" (*n* = 43, 19.55%) and "organization and logic" (*n* = 46, 20.91%) were also pointed out as shortcomings.


**Table 2.**

*Number of comments given in the self-evaluation.*

It is noteworthy that some participants pointed out both strengths and weaknesses of the essays. These results suggest that students were good at finding the strengths of their writing. However, writers seemed to find it challenging to analyze and identify problematic areas of writing, especially with regard to the global issue of content by themselves. This may be because reading self-written texts critically requires critical and objective views.

#### **3.3 The relationship between essay scores and views on peer feedback**

The results of the questionnaire analysis revealed that 89 students (72.95%) found peer feedback very useful and 19 participants (15.57%) found it useful, whereas 14 participants (11.48%) regarded it as not useful (**Table 3**). These results indicate that the majority of participants (88.52%) had positive views on peer feedback. This means that the participants thought peer feedback was worthy and they benefited from this experience in revising their essays. This finding is in accordance with those of previous studies [2, 20].

However, as shown in **Table 3**, some participants found peer feedback not useful regardless of the essay groups; the high-graded group had eight participants (19.05%) who viewed peer feedback as not useful, which was more than four (6.35%) and two participants (11.76%) in the medium- and low-grade groups, respectively. These findings suggest that as writing expertise increases, writers do not tend to find peer feedback useful.

To determine the association between essay scores and writers' views on the usefulness of peer feedback, a chi-square test was conducted, and the result did not show a significant difference (χ<sup>2</sup> (4) = 6.006, *p* > .199). This result indicates that the essay scores are not associated with the participants' views on the usefulness of peer feedback. Similarly, a Pearson's correlation coefficient showed that there was no correlation between essay scores and the recognition of the usefulness of peer feedback (*r* = .159, *p* = .080). Although the statistical analyses indicate that essay scores do not correlate with the view on peer feedback, the participants' comments are worthy of investigation. The next section details the analysis of comments that reflect peer feedback.

#### **3.4 Co-occurrence network of positive comments on peer feedback**

Analysis of the participants' comments regarding the usefulness or unusefulness of peer feedback revealed 125 positive and 39 negative comments. Among the 122 participants, 14 provided both positive and negative comments on various aspects of peer feedback.

Positive comments were analyzed using a co-occurrence network, and the results are shown in **Figure 1**. As shown in **Figure 1**, the participants' positive comments made seven communities, which are shown as subgroups. The most central community (subgroup 1) is related to writers' awareness raising toward problematic areas of their writing from an objective point of view. In other words, the writers had positive views on constructive comments given by their peers. According to **Figure 1**, it is implied that peers pointed out problematic areas critically and it was received positively. Furthermore, the relationship between reading and writing is linked and noticed through peer feedback, as the following comment shows:

*Peers' comments enable me to notice things that I was not aware and reading others' writing made me find how to improve my essay. (18E1-3)*

**Degree of usefulness Essay score Total (%) High (%) Medium (%) Low (%)** Very useful 26 (61.90) 49 (77.78) 14 (82.35) 89 (72.95) Useful 8 (19.05) 10 (15.83) 1 (5.88) 19 (15.57) Not useful 8 (19.05) 4 (6.35) 2 (11.76) 14 (11.48) Total 42 63 17 122

*Approach to Writing from Readers' Perspectives: Incorporating Self-Evaluation, Peer Feedback… DOI: http://dx.doi.org/10.5772/intechopen.100308*

#### **Table 3.**

*Students' views on the usefulness of peer feedback.*

#### **Figure 1.**

*Co-occurrence network of positive comments on peer feedback.*

By reading peers' essays, the writers were exposed to peers' writing and learned the features of good essays, which helped developing critical reading skills for their writing [7, 16].

The second community (subgroup 2) is associated with peers' evaluations and readers' expectations. Having the essay evaluated by peers encourages writers to consider their writing from new perspectives, and peer feedback allows writers to make sure that their ideas in the essays are conveyed to readers.

The third community (subgroup 3) is related to the importance of absorbing peers' opinions and feedback comments in their writing. More precisely, 16 students mentioned that they had some concerns about their writing, but peer feedback helped them to resolve the problems or concerns. Resolving writers' concerns and reducing their writing anxiety have been reported as advantages of peer feedback [10]. Since this community had the essay topic "robot" as the keyword, especially those who wrote about it, tend to think this way.

The fourth community (subgroup 4) focuses on the revision and improvement of essays. In this community, feedback comments seemed to help writers understand how to revise their essays, as the following participant's comment shows:

*I was able to make my essay better because peers pointed out things from the reader's perspective that I did not notice. I incorporated peers' opinions into my essay when revising it, and even when I did not accept a suggestion, I reconsidered my writing and thought about the reason I decided not to take it. Then, I got to know my essay deeply and considered why I wrote it the way it was written. (19E1-92)*

The writer's ownership is clearly presented, carefully selecting advice to take. This revision behavior is regarded as a characteristic of advanced writers [2]. It is important that students understand that they are decision-makers and that they do not need to take every advice and suggestion from peers when they come up with a better way to revise or when they think points pointed out should not be revised.

The fifth community (subgroup 5) is related to the amount of advice the writers received from peers. They appreciated a lot of advice or feedback from a third-person's point of view. This means that peers succeeded in pointing out or suggesting things that writers did not realize, which helped them revise the essays and learn new things.

The sixth community (subgroup 6) has only two nodes, namely, "advice" and "precise," which implies that the writers were satisfied with peers' concrete advice and suggestions. Making specific, precise comments is key to successful feedback [14]. Presumably, the instructions given to the participants before peer feedback may have played a role in encouraging them to give advice and suggestions as detailed as possible.

Similarly, the seventh community (subgroup 7) also has two nodes: "explanation" and "occupation". These are related to another essay topic, in which the participants were asked to choose their job of interest and explain its features. Because all writers chose their occupation as an essay topic, not everyone was familiar with the selected occupation. Therefore, from the reader's perspective, readers seemed to point out the lack of explanation or ambiguity of certain expressions about the occupation chosen.

Although communities 5, 6, and 7 are independent without a strong connection to the other communities, some participants pointed out more than one positive comment on peer feedback. As shown in **Figure 1**, participants' positive views on peer feedback were demonstrated through various aspects of peer feedback.

#### **3.5 Co-occurrence network of negative comments on peer feedback**

**Figure 2** visualizes the negative comments received by the participants. Compared to positive comments (*n* = 125), the number of negative comments (*n* = 39) was substantially lower, which suggests that most participants had positive views on their peer feedback experience. However, it is important to pay attention to the participants' negative comments to identify the challenges the participants encountered during peer feedback.

The first community (subgroup 1) identifies that the participants wanted more comments or concrete suggestions from peers to improve their drafts. This implies that they could not benefit from peer feedback effectively because of insufficient feedback. Regarding this point, one participant requested a follow-up session between the writer and reader after peer feedback as follows:

*Approach to Writing from Readers' Perspectives: Incorporating Self-Evaluation, Peer Feedback… DOI: http://dx.doi.org/10.5772/intechopen.100308*

*Most of the comments I received were not concrete, and thus, I did not know how to revise (my essay) and I could not make use of them. I was particularly concerned that I couldn't use any citation in the body paragraphs of my essay due to the word limit. I hoped to know my peers' thoughts on which part I could possibly omit. I wished I had had the opportunity to directly ask the reader what I was concerned about after I received the comments. (18E1-14)*

Written feedback comments were occasionally not fully understood by the receiver. Therefore, it may be a good idea to give students a short follow-up session to exchange their opinions directly or ask questions freely to benefit from peer feedback effectively. A combination of written and oral feedback seems to enhance the interaction between students, which leads to providing and receiving more concrete suggestions.

The second community (subgroup 2) is related to a lack of critical opinions from peers. Some participants mentioned that they hardly received advice or suggestions and struggled to revise their essays as follows:

*Although my peer pointed out some parts (of my essay), most of the comments were*  ◎ *(very good), and honestly, it was not easy to find out how to revise the essay. (19E1-115)*

The lack of criticism in peer feedback comments was reported previously (e.g., [19]), and this seems to be one of the reasons students do not regard peer feedback as useful. The following participant suggests that anonymous feedback should encourage readers to provide more critical comments.

**Figure 2.** *Co-occurrence network of negative comments on peer feedback.*

*Since I didn't receive concrete opinions, I didn't know what is lacking (in my essay). As I was ready to receive harsh comments, I wanted more suggestions. As readers of the essay hesitate to make critical comments with their names shown, I thought that peer feedback in an anonymous way would work better. (19E1-101)*

This suggestion is reasonable because students may be hesitant or reluctant to give critical comments to their peers. At the same time, it is crucial that teachers encourage students to give constructive and critical comments because they are more beneficial for receivers than just giving praises, and they can learn more from each other by critically reading and evaluating essays [7, 16].

The third community (subgroup 3) refers to a lack of advice that receivers wanted to take and the difficulty in taking advice from peers. More specifically, while some participants received advice, they found challenging to reflect it on their essays. It is not negative if students carefully select which advice they should incorporate into their writing because it means that they consider the revision and act as decision-makers [14].

The fourth community (subgroup 4) is related to the feeling of pity when writers could not revise the essay based on feedback due to the word limit. In some cases, writers pointed out that the feedback comments were not comprehensible, and therefore, they could not make use of them.

The fifth community (subgroup 5) identifies the struggle to improve the essay due to a lack of suggestions. The following participant points out the limitations of peer feedback by novice writers:

*I thought that even if those who have not been used to academic writing evaluated each other's essays, we could make only a little progress (in revisions). (18I-56)*

It is true that some students may not be able to provide useful comments and suggestions, and that especially less-skilled writers tend to struggle to make comments on global issues and suggestions [19]. However, experiencing a reader's perspective is helpful in revising the essay while critically reading and evaluating the peer's essay [7]. Furthermore, more coaching is necessary before and during peer feedback sessions [10].

The sixth community (subgroup 6) is associated with an insufficient number of critical comments from peers and difficulty in revising based on peers' comments. Some comments from peers may not be always precise and comprehensive.

*Peers' comments were not appropriate and abstract, and therefore, I did not know what to revise in my essay. (19I-57)*

Providing appropriate and concrete advice is not easy for novice writers. However, even if they receive inappropriate advice or suggestions they disagree with, they can make a decision about revisions. In this sense, peers' feedback comments allow writers to consider their essays actively even when they decide not to follow the advice they received.
