Writer-Reviewer Anonymity in EFL Peer Review

By Khaled El Ebyary

Department of Curricula and Methods of Instruction

Damanhour University, Egypt

Writer-Reviewer Anonymity in EFL Peer Review

Khaled El Ebyary

Damanhour University, Egypt

Abstract

The present study investigated the impact of writer-reviewer anonymity in the context of an academic writing course. Data collected for analysis in this study included data from two questionnaires, students' compositions and peer review comments. Embracing a quasi-experimental design, 34 undergraduate EFL students were matched in two homogeneous groups (i.e. anonymous and identified). Each group contained reciprocal dyads (matched with a similar ability peer), and provided either anonymous or identified peer feedback on each other’s compositions. The study aimed to examine a) the change, if any, in EFL students’ perceptions of peer feedback practice due to anonymity, b) the impact of anonymity, if any, on the amount of written work and the number of peer comments produced, c) the focus of peer comments between anonymous and identified dyads and d) the extent to which anonymity, or lack of it, would encourage/discourage reviewers’ criticality in their general evaluative comments and how writers’ responded to such evaluative comments. Major results indicated that anonymity positively affects length of written text, time spent in reviewing, number of comments in most language areas and a tendency towards criticality on the side of the reviewers.

Keywords: peer review, feedback, anonymity, criticality of peer review

ترکز الدراسة على اختبار استراتیجیة اخفاء هویة الأقران عند قیامهم بنشاط مراجعة الکتابة باللغة الإنجلیزیة وأثره على اتجاهات الطلاب ونوع المراجعة، وجودتها من حیث الکم والکیف. وقد تبنت الدراسة تصمیم البحث شبه التجریبی، وشملت عینة الدراسة ٣٤ طالبا فی قسم اللغة الإنجلیزیة بکلیة التربیة جامعة دمنهور تم تقسیمهم إلى مجموعتین؛ تلقت إحداهما تعلیقات دون معرفة هویة أقرانهم، أما المجموعة الأخرى فقد تلقت تعلیقات بالطریقة التقلیدیة. وتوصلت الدراسة إلى أن استخدام استراتیجیة اخفاء هویة الأقران له تأثیر إیجابی على جودة مراجعة الأقران مجهولی الهویة ومیلهم إلى النقد والإسهاب.

Introduction

Experts in the area of feedback in foreign/second language instruction (e.g. Ferris, 2003; Hyland & Hyland, 2006) suggest that the emerging significance of feedback came to the forefront of educational research during the 1970s when the learner-centred approach in education and the process approach in writing instruction were commonly used in North American L1 composition classes. In fact,  the process approach, describes Muncie (2000, p. 47), attempts to change the focus away from ‘an endless stream of compositions assigned by the teacher, written by the learners, handed in for marking by the teacher, handed back to the learners, and promptly forgotten by them as they start on the next assignment’ towards what Hyland and Hyland (2006, p. 83) describe asa learner-centred classroom in which feedback is seen as an important ‘developmental tool moving learners through multiple drafts towards the capability for effective self-expression’. In this respect, Hattie and Timperley (2007, p. 81) describe feedback as ‘a consequence of performance’ where an agent (e.g. teacher, peer, book, parent, self, experience) would provide information about one’s performance or understanding.

In an attempt to increase the opportunities for learners to receive feedback on their written work before producing a final version, researchers gave more attention to different feedback sources in addition to the teacher. Airing concerns about the relationship between peer review and learners’ writing ability is a path well-trodden in the area of foreign/second language research. While early research provided fluctuating evidence about the impact of peer review, recent empirical studies seem to suggest positive indications in support of the use of peer review in EFL writing classes. However, research identified a number of concerns, one of which is peer bias due to friendship, interpersonal relationships, sex, race, or individual inclinations. Such research suggests that these factors may influence the quality of feedback provided as peer reviewers can be afraid of being wrong or critical of friends’ written work. This gave rise to suggestions that anonymity of writers and reviewers can benefit peer review practice in EFL contexts.

This study focuses on peer feedback which, in a general sense, refers to the activity of providing and/or receiving written and/or oral feedback comments from one classmate or more on a peer's writing. As classified by Lai (2010), an overview of research in the area of peer feedback in EFL refers to three dimensions. These are a) perception of peer feedback, b) peer feedback processes and procedures and c) the product of peer feedback practice (i.e. the impact on L2 writing skills). This study examines a) the change, if any, in EFL students’ perception of anonymous vs. identified peer feedback practice, b) the impact of anonymity, if any, on the amount of written work and the number of peer comments produced, c) the focus of peer comments in anonymous and identified dyads and d) the extent to which anonymity, or lack of it, would encourage/discourage reviewers’ criticality in their general evaluative comments and how writers’ responded to such evaluative comments.  

Literature Review

Peer feedback defined

In the literature, the term peer feedback is generally used synonymously with peer review, peer evaluation, peer response, peer responding, peer assessment, peer editing, peer interaction and peer collaboration. Definitions of the term ranges from information provided to a peer (Topping, 1998) to reflective and evaluative discourse to improve writing (Strijbos & Sluijsmans, 2010). A detailed description of such practice by Liu & Hansen (2002) refers to:

‘the use of learners as sources of information and interactants for each other in such a way that learners assume roles and responsibilities normally taken on by a formally trained teacher, tutor, or editor in commenting on and critiquing each other’s drafts in both written and oral formats in the process of writing’ (Liu & Hansen, 2002, p. 1).

Furthermore, Yu and Lee (2016) explain that, as a term, peer feedback refers to the activity as a whole, comprising both the process and the product of such an activity. Pedagogically defined, Hansen and Liu (2005) explain that in the peer feedback situation one peer is trained to provide written or spoken comments on his/her peers’ initial drafts. Equally, Li (2017) describes peer feedback as a learning tool in which the student assumes the role of an reviewer and/or a reviewee. In addition, terms like peer comments or reviewer comments are used to refer to the actual comments provided by a peer assuming an assessor role, while the writer’s response to comments is sometimes known as revision behaviour or reaction to peer feedback.

The benefits of peer review practice in L2 contexts have been acknowledged in the literature (Tsui & Ng, 2000; Lai, 2010; Gielenab, Topsa, Dochya, Onghenac, & Smeetsa, 2010; Yu & Lee, 2016; Djuddah, 2017; Zhao, 2018; Ruegg, 2018). Some of these benefits were summarized by Edwards (2013) as developing understanding of the nature of writing, assessment,  audience and what high quality work is. Tutors using peer review can organize it in various ways some of which are:

a)      L2 learners can collaborate as either dyads (i.e. pairs) or small groups,

b)      peer comments can take the form of oral or written feedback,    

c)      reviewers can use their knowledge to provide comments or use external resources (e.g. adhere to a checklist on certain language area(s), use internet resources when possible and so on).

Issues like matching pairs, language proficiency of the writer/reviewer, peer bias, trained versus untrained reviewer and others have come under scrutiny in empirical research. Over time, this has led some researchers to suggest that ‘there does not appear to be consensus on how to optimally match students during the peer feedback process: with same-ability peers (homogeneously) or different-ability peers (heterogeneously)’ (Huisman, Saab, van Driel, & van den Broek, 2017, p. 1434). Other researchers in the area of peer review have shown interest in the writer-reviewer anonymity, which is a main focus in this study as this researcher will explain in a subsequent section. As peer review involves collaboration and interaction between two classmates or more, recognition of the presence of interpersonal variables is well-established. Researchers such as Rotsaert, Panadero, & Schellens (2018), Li (2017), Panadero (2016) and Vanderhoven, Raes, Montrieux, Rotsaert, & Schellens (2015) have indicated the potential negative impact of interpersonal beliefs on output due to variables such as value of friendship between peers, self-distrust in one’s ability and fear of dissatisfaction when assigning lower marks or negative feedback (i.e., recrimination). Nonetheless, the literature on peer review has documented evidence of the benefits of such practice such as enhancing motivation (Brown, 2004), increasing the quantity and timeliness of feedback (Gibbs, 1999) and improving learning (Black & Wiliam, 1998).

Theoretical underpinning

Research on peer feedback based itself on a number of theoretical frameworks including, but not limited to, process writing theory, collaborative learning theory, Vygoskty’s sociocultural theory (SCT) and the interaction hypothesis. In this respect, Zhu (1995) explains that enthusiasm of peer review practice finds strong support in theories that emphasize the social nature of language, thought, writing, and learning. A detailed analysis of each of these theoretical underpinnings is beyond the scope of this research work. However, discussion of Vygoskty’s SCT is required as it provides the main theoretical framework for this work. As an ingrained theory in psychology and education, SCT provides a robust account of mental development (Lantolf, 2008). Lantolf (2008) explains that according to this Vygotskian line of thought, the source of knowledge building is not to be sought in the mind, but in the social interaction between individuals of differing language abilities. Three major principles are significant; a) language is the central tool of thought, b) social interaction is the foundation of learning, and the zone of proximal development (ZPD) is the crucial activity space in which learning happens, and c) MKO (i.e. more knowledgeable other) refers to someone who has more knowledge or skills than the learner, particularly in relation to the task at hand. To Vygotsky (1978, p. 86), ZPD is ‘the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem-solving under adult guidance, or in collaboration with more capable peers’. In other words, ZPD is the difference between what a learner can do without help from a MKO and what he/she can complete with the support of a MKO. Hence, peer feedback is an opportunity for social interaction with peers (possibly an MKO) and this in itself, following Vygotsky’s line of thought, is an effective way of developing writing skills and strategies.

Peer feedback research and anonymity

More recent research seems to suggest a positive indication to support the use of peer feedback in EFL writing classes (e.g. Yang, Badger, & Zhenc, 2006; Lai, 2010; Yu & Lee, 2016; Dijls, Brummer, & Kostons, 2018). The pedagogical implications of such research built on recommendations of previous researchers that EFL tutors integrate peer review into their classroom practice on the basis that it helps L2 learners to become good writers, readers, and collaborators (e.g. Berg, 1999; Paulus, 1999). However, researchers have uncovered various challenges inherent in peer review practice. One of these can be termed the distance between the writer and the reviewer, or the writer-reviewer relationship, and the likelihood of peer reviewers being biased for various reasons; for example, because of friendship with writers; sex; race; interpersonal relationships; or individual preferences (Carson & Nelson, 1996; Ghorpade & Lackritz, 2001; Nilson, 2003). Rollinson (2005) describes interpersonal relationships as student characteristics, which, he adds, can be a problematic aspect of peer feedback. Thus, in certain contexts, cultural and social variables might negatively impact the peer review process between classmates (Topping, 1998).

An interesting area of debate, therefore, is the issue of anonymity between writer and reviewer, and if this would have an impact on the peer review process and product. To Lu and Bol (2007) ‘anonymous peer review refers to a kind of peer review condition in which both reviewers and reviewees are kept unknown to one another’. The debate for and against anonymity in peer review has been governed by two major psychological concepts which exert pressure on the review process: ‘deindividuation’ and ‘social loafing’. Lu and Bol (2007) explain that anonymity in peer review is based on the concept of ‘deindividuation’ in psychology, which, as cited in Jessup, Connolly, & Tansik (1990), is described as urging individuals within a group to ‘stop thinking of other members as individuals and feel that they cannot be singled out by others’ so that any subliminal constraints are minimized to enable engagement in behaviour that they would not customarily engage in. Lu and Bol (2007) therefore, suggest that anonymous peer review can enhance ‘deindividuated behavior’ as, they state, ‘the assessment system separates people from one another and detaches individuals from their contributions’.

The other psychological concept is social loafing, the predisposition of an individual to make less effort when they are part of a group work. Such predisposition could be on the physical or cognitive sides. Thus, the collaborative and social nature of the peer review process exert all sorts of pressure on participants. This pressure has been described in the literature as interpersonal beliefs (Raes, Vanderhoven, & Schellens, 2013; Panadero, 2016). The acknowledgement of the existence of interpersonal variables in peer review is central because the burden students carry in the process might influence how they perceive the value of the process (Li, 2017). Furthermore, although Rotsaert, Panadero and Schellens (2018) add that constant anonymous peer review can eventually prevent interactive feedback dialogues, these authors, as well as Nguyen (2016 ) still emphasized that anonymous peer review creates a practical and manageable    peer assessment setting and plays a fundamental role in increasing students’ participation in L2 writing.   

An overview of previous studies in the area indicates that various researchers have been investigating the effect of anonymity on EFL writing  and positive  effects  were reported (e.g.Lu & Bol, 2007; Guardado & Shi, 2007; Jin & Zhu, 2010; Lu & Law, 2012; Li, 2017). The conclusion drawn from studies conducted by Tuautmann, Carlsen, Yalvac, Cakir, & Kohl (2003) and Haaga (1993) on the use of anonymous peer review suggest that it is an effective approach in improving students’ written work. Using a post-test to compare an experimental, and a control, group both involving 44 eleventh-grade students in Singaraja, Adhiyanti, Marhaeni, & Kusuma (2018) examined the influence of anonymous peer review on students’ writing ability. Findings suggested a significant difference on writing ability between the eleventh-grade students who obtained feedback from anonymous peer assessment and those who obtained feedback from conventional assessment. Li (2017) conducted a quasi-experimental study to examine the influence of anonymity and training on 77 learners’ performance and perceptions in peer review. The study involved three groups. This researcher refers to these groups as Identity Group (participants’ identities revealed), Anonymity Group (participants’ identities concealed) and Training Group (identities shown but training provided). Findings indicated that Anonymity Group and Training Group outperformed the Identity Group on projects, but in relation to perception the Training Group valued peer review practice.

Coté (2014) carried out a study in which he was the researcher and the class teacher. The study took place in an EFL expository writing class at the American university in Madrid, Spain. Adopting a combination of interviews, observation, and documents analysis, the study involved 25 participants  (age between 18-25)  whose paper-based TOEFL scores ranged from 443 to 593 and therefore, attended weekly 3 hours intensive EFL advanced writing class. L1 of the participants included Arabic, French, Chinese and Spanish. Results of the study suggested a tendency to accept peer comments in the anonymous mode.

To examine the influence of anonymity on criticality in peer review context, Zhao’s (1998) research created two comparable situations in which anonymous authorship and reviewing were manipulated. In the first situation, the personal information of both writers and reviewers was removed from the scripts, and both parties were guaranteed that writing and reviewing would be completely anonymous. Conversely, the second situation involved identifiable authorship and reviewing in which names of both writers and reviewers were left on scripts. Findings reported that peer comments given in the anonymous situation were more critical than those made in the identifiable one. Similar results on criticality of anonymous peers were reported Valacich, Dennis, & Nunamaker (1992) and Connolly, Jessup, & Valacich (1990) and others.

In summary, while research has indicated that identified peer assessment practice can be subject to interpersonal subliminal restraints, anonymous peer review can provide L2 learners with secure and scaffolded opportunities of learning.  

The Current Study

In the light of the peer review literature, the aim of this study was to examine a) the change, if any, in EFL students’ perception of anonymous vs. identified peer feedback practice, b) the impact of anonymity, if any, on the quantity of written work produced by the writer and the number of peer comments produced by the reviewer, c) the focus of peer comments between anonymous and identified dyads and d) the extent to which anonymity, or lack of it, would encourage/discourage reviewers’ criticality in their general evaluative comments and how writers’ responded to such evaluative comments. In so doing, the following five main research question were formulated:

1. 1.      What is the participants’ past experiences, if any, with peer review and what are their perception of such practice?
2. 2.      What is the impact of anonymity, or lack of it, on the amount of text produced by writers’, the number and length of comments given by reviewers and time spent on providing feedback?
3. 3.      What is the focus of feedback comments provided by anonymous/identified dyads?
4. 4.      To what extent does anonymity, or lack of it, encourage/discourage criticality on the part of reviewers? And what does writers response to criticality look like?
5. 5.      What is the impact of anonymity, or lack of it, on participants’ perceptions and preference of peer review practice?

Method

Participants and the Research Context

Participants in this study (N=34) were first year students at the English Section in a faculty of education in Egypt. All participants happened to be female students. They enrolled on an 8-week academic writing course which is intended to provide opportunities for students to develop their English writing skills including paragraph writing, formal and informal styles, short essays, cover letters and different genres. While instructors use students’ written work for formative assessment purposes, students receive very little feedback on their written work (mainly paragraphs on different topics at the time this study was conducted), largely because there are large numbers of students, writing many paragraphs, and only one class teacher. Hence, there is a need to use peer review, especially with the course objectives adopting a writing process approach. All participants obtained their secondary school certificates from state schools. 

Study Design

Pairing up and training participants

As mentioned earlier, the present study focused on peer review anonymity in the context of an academic writing task involving first year EFL participants. In peer review research, mentioned Rollinson (2005), a number of important decisions with regard to the setting up of the group has to be made. The study adapted a quasi-experimental design in which participants were randomly divided into two main groups. One group was assigned to provide/receive identified peer review (i.e. the identity of the writer and the reviewers was known to each other) and the other group provided/received anonymous peer review (i.e. the identity of the writer and the reviewers was unknown to each other). Within the anonymous group, participants were matched in 8 homogeneous reciprocal dyads (matched with a similar ability peer). These are labelled dyad A, B, C, D, E, F, G & H. Within the identified peer feedback group, 9 homogeneous reciprocal dyads were involved and they were categorized as I, J, K, L, M, N, O, P & Q. In this respect, van Zundert, Sluijsmans & van Merriënboer (2010) differentiate between a student’s ability to provide peer feedback (e.g., ability to understand and use criteria) and a student’s ability to complete the task  (e.g., writing skills). As mentioned above, peers in both groups were matched on the basis of their task related ability. In other words, the Department of English at the Faculty of Education in the research context requires certain scores in Secondary School Certificate in general and in English as a subject in order for students to enrol at first year. Participants’ scores were an evidence of homogeneity in both the anonymous and the identified dyads. Pairing up participants was based on an assigned pair protocol in the anonymous and identified groups. This helped in eliminating a typical hurdle reported in the EFL literature (see Hyland & Hyland, 2006) such as inability of less proficient learners to produce useful feedback that their more competent peers anticipate (Hanrahan & Isaacs, 2001). A static writer-reviewer dyads protocol was adapted for the writing task involved. This study involved only one task as reiteration would have compromised anonymity (as this researcher will explain in a subsequent section).

As participants in this study were first year students, a peer review training workshop was conducted before students embarked on any peer review practice in order to coach participants in providing and receiving useful peer feedback. This was based on recommendations in put forward in the literature that training enhances positive impact (e.g. Min, 2006; Diab, 2009). The training session started with the introduction of the concept of peer review and its function. The training was adapted from the work of Lam (2010) and Min (2005) whose procedures for peer review were adopted. Participants were trained on how to conduct peer review on a mock paragraph using Yim et al’s (2018) examples of focus of feedback (see Appendix D).

The course, the writing task and writing procedures

The writing course for first year students in the research context of this study is intended to provide opportunities for students to develop their basic academic English writing skills including short paragraph writing using formal and informal styles, cover letters and different genres (e.g. argumentative). Generally, a student-centered approach to teaching writing features is the approach adopted in the course description. This approach enhances student-writers’ reflection on their own writing process with the students ultimately producing their own strategies for prewriting, drafting and rewriting. When this approach is adopted, providing feedback is perceived as an essential instrument for the development of foreign language writing skills. However, students in the target context get little teacher feedback on their written work (mainly paragraphs), primarily because there are too many students writing tasks every week, and usually one tutor, with very little time. In such a classroom, the value of teacher feedback falls short and hence, the use of peer feedback is needed. As the nature of writing tasks is commonly addressed is studies on writing, discussion of the writing tasks involved in the course was deemed important to this study, particularly when the literature emphasizes the relationship between feedback and the tasks at hand (e.g. Hyland, 2003). The course tutor had a list of writing tasks (see Appendix A) for weekly assignments. Each student was offered the list and he/she would choose a topic for his/her weekly assignment provided that no single topic is chosen more than once. The tasks were chosen on the basis of familiarity and interest.  

Instruments

The present study used two questionnaires (i.e. an 11 item pre-activity questionnaire and a 19 item post study one) and students’ written documents. The two questionnaires were a pre-activity questionnaire and a post-study questionnaire. The pre-activity questionnaire was used to collect demographic data in addition to information about participants’ views with regard to peer review. The specific aims of the questionnaire were a) providing an understanding of participants’ past peer review experiences, if any, b) recognising willingness to participate, any predispositions towards peer review practice and the role of peers in improving one’s learning and c) unveiling participants’ expectations of peer review practice before partaking in the current study. This questionnaire was mainly based on a Likert-scale in addition to some open-ended items (see appendix B). A post study questionnaire was also used in this study (see appendix C). Its aim was to collect data on participants' perception and preferences of anonymous vs. identified peer feedback practice. The design of both the pre-activity and the post study questionnaires was informed by Sato (2017), Sato & Viveros (2016), Cote (2014) and Levine, Oded, Connor & Asons (2002).

It is worth mentioning that data collected from the pre-activity and the post activity questionnaires were anonymous at the point of collection. This was meant to reduce social desirability bias, which is defined as a tendency among participants to provide information that would make them look good or data that might pass as acceptable/expected in their context rather than their genuine views (Fisher, 1993; Mitchell, 2010; Krumpal, 2013). Unlike the pre-activity questionnaire, the post study questionnaire was administered with each group separate from the other so that data from anonymous dyads can be differentiated from the identified ones (see Appendix C.a and C.b). Data from the students’ written work were analysed using content analysis. Data came from coding the comments made by reviewers and the responses provided by reviewees in both anonymous and identified groups. Yim et. al’s (2018) coding scheme was adapted.

Study Procedures, Data Analysis and Ethical Considerations

The current study was carried out as part of the writing course for first year students in the English Section. Week one involved obtaining consent from the teacher and the participants in the research context. Week two week two involved the training workshop referred to earlier in this study and week three witnessed the administration of the pre-study questionnaire. In week four, grouping and pairing up dyads took place followed by the peer review process itself with students each in their assigned anonymous or identified dyad. The post study questionnaire was administered in week four. The data collected for analysis in this study included data from the questionnaires and students' paragraphs. Descriptive analysis was undertaken in analysing questionnaires' data, and qualitative data were thematically approached. The peer review process involved 34 first year students’ paragraphs written for a descriptive task (see Appendix A). The coding scheme used to analyse the focus of peer comments was informed by the work of Yim, Zheng, & Warschauer (2018); Yim, Warschauer, Zheng, & Lawrence (2014); Ellis (2009); and Stanley (1992) (see appendix D). Although some of these schemes (e.g. Stanley’s, 1992) focused on reviewers’ responses and writers’ reactions to the feedback, only reviewers’ responses categories were mainly analysed here. Writers’ responses were considered only in terms of reacting to reviewers’ overall evaluative comments. In other words, the coding scheme included two tiers of coding, the first of which aimed at categorizing the focus of feedback comments provided by anonymous and identified dyads (i.e. areas of revision evaluators focused on) and the second tier looked at a) the reviewers' overall evaluation of their peers' written work (which was used to determine the level of criticality in anonymous and identified reviews) and c) writers' reaction with the comments provided.

Ethical Considerations

The present study followed ethical considerations at all stages of data collection and data analysis. Information sheets, in which participants were informed about the aims and the procedures of the study were distributed. Consent forms were circulated in week one and were signed by all the participants. As participation was voluntary, participants were made aware they could withdraw their data from the study at any time of data collection and up to two weeks after data was collected. Questionnaire data collection was anonymous, but written work was identified to the researcher in all groups. However, participants were assured of confidentiality and use of pseudonyms to replace their real names. Furthermore, participants were informed that data would be used merely for research purposes.

Results

The overarching aim of the study focused on investigating the impact of anonymity in peer review on EFL students' perception, amount of written work and comments produced, focus of peer comments and whether or not anonymity would encourage/discourage reviewers’ criticality. Results are presented in the light of the five research questions outlined earlier.

Participants’ past experience & perception of peer review practice

The first research question in this study aimed at creating a profile of participants’ experiences and perceptions of peer review practice prior to engaging in the study. The data used in building such a profile came from the pre-activity questionnaire.

Past experience

In relation to past experiences, data analysis revealed a lack of experience with peer review practice as 25 participants responded negatively to the question ‘Have you ever done peer review in an English course before?’ (see figure 1). Although a few participants (N9) answered positively, this researcher claims that such answers were based on misunderstanding of the concept peer review practice per se. Answers to the open-ended item from some of these participants showed confusion and lack of understanding of what peer review really meant. These students had participated in a type of teacher-led peer marking activity in private tutoring while at secondary school as a way of overcoming the marking load of secondary school teachers. In such an activity, learners swapped their completed formative assessment tasks with their nearest peer. The teacher would then provide the answers orally and students marked, and scored, each other’s work. This result shows that a few participants still could not differentiate between this peer marking practice and peer review practice in spite of the training provided to them prior to the study. Furthermore, data from the pre-activity questionnaire also showed that students were often in search for feedback opportunities in order to improve their writing. In this respect, participants were asked about the possibility of other family members reading their written paragraphs before they handed them in to their secondary school teachers. While it is true that some participants sought outside help to check their writing, they often saw this task as the job of their private tutors (see figure 2). This could have been governed by their eagerness to score better in their final exams given that access to top universities is based solely on achievement in final, often high stakes, tests. Having said that, participants expressed beliefs in their open-ended explanations that it is the teacher who should provide good opportunities for scaffolded learning in the sense that almost all of them agreed that it is the teacher's responsibility help students to learn English.

Participants’ perception

The second part of the first research question aimed to unravel participants’ perception and expectations of peer review. Data collected from the pre-activity questionnaire aimed to identify participants’ perception of a) own and others’ capabilities of providing feedback, b) potential role of peers in improving one’s learning and c) any predispositions towards peer feedback.

One area of concern was whether or not participants notice the written or spoken errors of their peers in any classroom activities. The pre-activity questionnaire collected data on whether or not participants notice peers’ spoken or written language errors in any classroom activities. The majority of participants’ answers indicated awareness of peers’ errors (see figure 3), but this does not necessarily mean that all their observations of errors were correct. In response to their behaviour in relation to such noticing, almost all of them explained that they often keep this to themselves. Supported with this researcher’s understanding of the research context, this can be seen as justifiable as these participants attended a highly examination-oriented schooling system which was based on competitiveness rather than collaborative learning. This researcher assumes that such constraints could have implications for peer review practice. Interestingly, further analysis showed participants were aware that some peers were better language users and some of them sought answers for that. Their open-ended responses were indicative of such awareness. Yet, such responses showed that such awareness was geared more towards better language users. One student explained this view in his/her own words which translated as:

 ‘I know X [name of peer omitted] often participate in oral communication with teachers and I noticed her pronunciation in which she tried hard to imitate the American accent. I always wanted to ask her what she does …also in the composition class, the teacher sometimes uses her composition as an example’.

Another student said:

‘…my friend and I graduated from the same secondary school and we got exactly the same mark in the secondary school certificate …and she often scores better in composition and the teacher seems to be pleased with her. I’m generally embarrassed to ask her about how she studies…’.

It is worth mentioning here that very few participants provided an answer to how comfortable students were about having their work corrected by a classmate. Although data on the matter would have provided a snapshot about a number of things including how they perceived peers’ abilities in providing peer review, lack of responses was understandable as a) these participants had never had any experience with genuine peer review practice and b) they were familiar with high stakes assessment schooling system that based itself on competition and individual differences rather than collaborative learning which is characteristic to peer review practice.

More detailed answers were provided in a further set of questions in the pre-activity questionnaire. A 3–point Likert scale that required participants to rate their dis/agreement with eight statements that helped in understanding participants’ perception of, and attitudes toward, peer review practice.

As seen in figure 4, all participants liked receiving feedback, and 29 thought it can be used in their classrooms. Some participants (N=17) had confidence in their language ability to provide feedback, but fewer participants showed confidence in peers’ abilities (N=10). Having said that, a general feeling of uncertainty among participants was revealed in relation to willing to provide feedback (N=23); self-confidence in language ability (N=11); confidence in peers’ abilities (N=18); and the potential of peer feedback in improving their writing skills (N=19). Such uncertainty was justifiable as feedback practice is new to these participants; moreover, the impact of their exam-oriented school system encourages competitiveness rather than collaborative learning.  

The impact of anonymity

The second research question in this study was concerned with the impact of anonymity, or lack of it, on the amount of text produced by writers’, the number and length of comments given by reviewers and time spent on providing feedback. Answers to the second research question were based on the analysis of students’ texts and data from the post study questionnaire. Results on each theme are introduced in the following paragraphs.

Writing and response length

Data analysis was based on a simple word count of writers’ written work versus reviewers’ comments in both the anonymous and the identified peer review groups (see appendix E). In other words, the second research question was intended to explore whether or not writing for an anonymous reviewer (as opposed to an identified reviewer) and reviewing an anonymous writer (as opposed to an identified writer) would extend the amount of commentary beyond what would be normally expected in traditional identified peer review settings. Here, Independent Samples T-Tests (i.e. between subjects design) were used to compare the means from the anonymous (condition A) and the identified (condition B) peer feedback groups in order to examine the effect of anonymity (the independent variable being manipulated) on length of writing (the dependent variable being measured). The examination of the impact of anonymity on length of written text shows the magnitude of the difference between the two conditions, with the anonymous peer feedback group having a higher mean (123.6250 ). However, the scores in condition A do not vary too much more than the scores in condition B. Put statistically, the variability in the two conditions is not significantly different (see tables 1, 2, 3 and 4).

Following the same line of thought, Independent Samples T-Tests (i.e. between subjects design) were used to compare the means from the anonymous (condition A) and the identified (condition B) groups in order to examine the effect of anonymity (the independent variable being manipulated) on length of peer comments (the dependent variable being measured). As indicated in table 3 of the Group Statistics box, results show that the mean for the anonymous group (condition A) is 43.5 while it is 20.4 for the identified group (condition B 2). The Sig. (2-Tailed) value here is .004 demonstrating that there is a statistically significant difference between the mean number of word counts in the peer comments for the anonymous and identified conditions, i.e. peer reviewers in the anonymous condition wrote significantly more extensive comments than peer reviewers in the identified condition.

Time spent in providing feedback

In terms of time spent in providing comments, participants in the anonymous and the identified groups were asked in the post study questionnaire to estimate the time they spent in providing feedback on their peers’ writing ( see appendix C). Anonymous reviewers tended to take more time than identified reviewers. In fact, 11 anonymous reviewers responded that it took them between 15 and 30 minutes to provide their comments when 11 identified reviewers took less than 15 minutes (see figure. 5). Although such data is based on self-report and is not linked here to the type and quality of the comments provided in either group, it is still indicative of the fact that anonymity encouraged reviewers to look more on the text. 

Impact of anonymity on feedback focus

The third research question in this study compared the focus of the feedback (i.e. areas of revision reviewers focused on) provided by anonymous dyads with those provided by identified ones. Based on the coding scheme, the major elements of feedback focus included seven language areas which were grammar, mechanics, word choice, writing conventions, content and organization. Only for the purpose of reporting results on peer review were these areas clustered into a) grammar, b) mechanics and c) others (where ‘others’ referred to word choice, writing conventions, content and organization).

In comparing anonymous and identified reviewers’ comments, the number of comments provided in each of the areas were aggregated. In the following paragraphs, findings on each dimension are presented. In terms of grammar, data analysis indicated that comments on grammar errors in both anonymous and identified reviewers followed a similar pattern. However, anonymous reviewers tended to provide more comments to their peers than identified reviewers. As seen in figure 6, anonymous reviewers gave more attention to conjunctions, subject-verb agreement, tenses, prepositions, missing words and articles correspondingly.

Also, identified reviewers focused more in their comments on the areas of conjunctions and tenses, but then gave equal attention to the rest of the areas (i.e. subject-verb agreement, prepositions, missing words and articles). Noun-pronoun agreement, redundancy and verb patterns were given less attention by anonymous reviewers and no attention by identified ones.

As for mechanics, anonymous and identified dyads followed a similar pattern of interest again. In this pattern, punctuation and spelling errors seemed to attract reviewers’ attention in both groups (see figure 7). Nevertheless, the number of errors detected by anonymous reviewers was consistently higher (26 punctuation and 12 spelling errors) as opposed to those detected by identified reviewers (9 punctuation and 4 spelling errors). Comments on capitalization were provided by anonymous dyads only.     

The third area of feedback focus looked at word choice, writing conventions, content and organization. Equal attention was given to organization by anonymous and identified reviewers, but the anonymous group focused more on word choice, content and conventions (see figure 8).

Impact of anonymity on reviewers’ criticality

The fourth research question in the current study examined a) the extent to which anonymity, or lack of it, would encourage/discourage reviewers’ criticality in their general evaluative comments on the written work of their peers, and b) the way writers responded to such evaluation. As part of the reviewing task, peers in both the identified and anonymous dyads were asked to provide an overall evaluation of the written work and writers were required to respond to such comments (see task description in the methodology section). This was intended to examine whether or not anonymity might encourage/compromise interpersonal factors which might hamper/impede reviewers’ ability/capacity to be critical when necessary.

Table 5 provides samples from the anonymous dyads. An overall observation of the comments made by the anonymous dyads refers to a tendency towards criticality, providing some sharp comments as seen in table 5.  Such comments were more direct and void of any positive reinforcement, emoji and language that might boost motivation (e.g. affective elements). While most writers receiving such comments from their anonymous peers generally showed no response, a few challenged the comments as seen in j in which the writer expressed disagreement with the reviewers’ comments and tried to provide reasons for such disagreement. Generally speaking, the anonymous feedback focused on problem identification and criticism, but provided no advice, explanation, solutions or praise.

Following the same line of thought, criticality was examined among the identified dyads. As seen in table 6, the general impression of the identified peer comments shows almost no criticality of the written work. Reviewers were keen to put forward positive comments in most cases and even reserved comments when reference to errors were made. Indeed, when a reviewer wanted to refer to errors detected, he/she assured the writer that his/her meaning was still clear. Furthermore, affective elements were clearly observable in the feedback provided. In a similar manner, writers were positive in their responses and seemed to accept it and occasionally a ‘thank you’ was provided.

The impact of anonymity on students’ perception

The fifth research question in this study was concerned with the extent to which both anonymous and identified peer review had an impact on participants’ perception and preferences of such practice. In reporting findings on this research question, data from section one of the post study questionnaire (see appendix C) reiterated the dis/agreement Likert-scale used in the pre-activity questionnaire (see appendix B). Similar to findings from the pre-activity questionnaire, aggregated figures from both groups show that all of the participants involved in the two groups (N34) liked the idea of receiving peer feedback and thought it could be used in their current classroom and could help improve their writing ability. However, unlike the uncertainty shown in the findings of the pre-activity questionnaire, the majority of participants a) liked providing feedback (N29), b) showed confidence in their ability (N27) and their classmates’ abilities (N24) to provide peer feedback, and c) thought peer feedback practice was valuable to revision (see figure 9).

The last research questions also aimed to find out differences, if any, in participants’ preferences of anonymous or identified peer feedback practice. As the administration of the post study questionnaire enabled differentiating responses from anonymous and identified groups (see instruments section), figures 10 and 11 compares preferences of dyads in

 each groups. As seen by participants, received peer feedback in both groups seemed to be focusing on local errors rather than global ones. In terms of preferences, while many participants in the anonymous group (N11) preferred providing feedback to unknown authors, far fewer (N6) wanted to receive anonymous feedback. Following the same line of thought, many anonymous dyads (N9) responded positively about providing critical feedback to unknown authors, but were uncomfortable in receiving anonymous critical feedback (see figure 11). As for the preferences of identified dyads, slightly more than half of the participants in this group (N9) preferred providing feedback to writers whose identities are known to them, and all of them (N18) preferred receiving feedback accounts to writers known to them. Similarly, criticality among the majority of identified dyads made them uncomfortable. 

Discussion and Conclusion

Results obtained in this study seem to agree with the literature in many respects, but it equally important to note that most results were based on self-reported data of participants whose views were likely influenced by the nature of the highly examination oriented context too. Such results suggest that EFL student value feedback that can help them improve their language ability and consequently, score better in their assessment. In peer feedback, they tend to prefer receiving, rather than giving, feedback, which agrees with results obtained by Gielenab et. al (2010). Yet, a need for teacher feedback was expressed and it is still viewed as more reliable than other sources of feedback. This claim finds support in Yang et. al’s  (2006) study that students tend to adopt feedback received from teacher rather than peers. An interesting point in the finding refer to participants’ tendency to participate in an ongoing peer review process in their heads by noticing the spoken and/or written errors produced by their peers even though they had not been asked to peer feed back. Again, this could be due to the general predisposition towards competition as these students often compare their marks to their peers.

One major area of interest in this study was the impact of anonymity on a number of issues. Results in this study seem to indicate that anonymity did not have particular influence on the focus of peer review as identified and anonymous reviewers were mainly interested in some local language areas, which agrees with Wu, Petit and Chen (2015) and Diab (2009). Yet, anonymity seemed to have encouraged reviewers to spend more time reading the text and to extend the amount of commentary beyond what would be normally expected in traditional identified peer review settings. In fact, peer reviewers in the anonymous dyads wrote significantly more extensive comments than peer reviewers in the identified ones. Furthermore, anonymity seem to encourage criticality which focused on problem identification and criticism and this agrees with results obtained in Lu and Bol’s study (2007), but criticality in the current study, as opposed to the identified mode, provided no guidance, explanation, solutions or praise. However, the majority of participants in both groups showed negative attitudes towards receiving critical feedback, which agrees with Edwards (2013) and Lu and Bol (2007). Thus, the current study has uncovered a further set of questions connected to why EFL students like receiving peer feedback, but don’t seem to want anyone to be critical, or too critical.

As a form of formative assessment, peer review research still needs to link itself to a number of areas in second/foreign language research. Partly because it has no high stakes consequences on learners’ future, several studies have reported the benefits of peer reviews in EFL writing classrooms. However, the extent to which peer review can have short/long term consequences on improving writing ability, i.e. washback on learning, is still to be investigated. Peer feedback research in EFL contexts seems to ask more questions than they answer ones. It often urges the question of what role(s) peer feedback can play. In fact, peer feedback, if appropriately used, can cut both ways. It is often useful if teachers use it as a kind of intermediate way of getting out of marking multiple drafts of written work in EFL contexts and they might also use it as a motivating strategy for students to try harder as they know that someone other than the teacher will be looking at it. Student like this sort of practice as it does not have any consequences for them and the anonymity even make losing face less risky. So, there is a bit of protectionism going on. It is an implicit reminder to EFL writers that every time they write, there will be a reader. This encourages them to think about someone outside of their heads. However, EFL teachers play quite an important part in how they set it up. The teacher has to cultivate a positive atmosphere in which mutual support is offered. Peer feedback can be very much like going round in circles if teachers are not aware of his/her students’ preferences and setting up ground rules about criticality.