Accounting for Change in Critical Thinking Components Mediated by Differential Effects of Paper-based vs. Web-assisted Feedback in Writing

Introduction

Criticality or critical thinking (CT) has been perceived in diverse manners. The consensus about the precise nature of criticality is little (Allen, Rubenfeld, & Scheffer, 2004; Fahim & Ahmadi, 2012). A number of senses about the nature of critical thinking has been proposed by different scholars. Klimoviene, Urboneine, and Barzdziukiene (2006) maintain that critical thinking is “making sense of our world by carefully examining our thinking and the thinking of others in order to clarify and improve our understanding” (p. 22). Also, Liaw, Huang, and Chen(2007) takes critical thinking into account as “the educational cognate of rationality”
(p. 32). Based on what Chance (1986, p. 6) postulates, critical thinking deals with the potential to scrutinize different issues, create and formulate viewpoints, provide proofs for them, “juxtapose, deduce, assess inferences, and find solutions”. Mayer and Goodchild (1990, p. 4) consider critical thinking as “an ongoing, rule-governed perception and assessment” of diverse issues. Scriven (2006, p. 2) perceive it as the cognitively principled way of “actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/ or evaluating information gathered from, or generated by observation, experience, reflection, reasoning, or communication as guide to belief and action”. According to Paul and Elder (2009) critical thinking is perceived as the ability of thinkers to take the responsibility of their own thinking process. This requires that they develop sound criteria and standards for analyzing and assessing their own thinking and routinely use those criteria and standards to improve its quality. Facione (2010) classifies critical thinking in six cognitive skills: interpretation, analysis, evaluation, inference, explanation, and self-regulation.

Lipman (1988) states that normal thinking and critical thinking are two different issues. Normal thinking is not complicated and principled, and it comprises merely the mental processes applied to solve problems or to make decisions. In contrast, critical thinking is more complicated and rule-governed, and it includes “skillful, responsible thinking that facilitates good judgment since it relies on criteria, is self-correcting, and is sensitive to context” (p. 39). A number of investigations can be found in terms of the impact of feedback on language proficiency (e.g. Bitchener & Knoch, 2010; Caffarella & Barnett, 2000; Guenette, 2007; Truscott, 2007). However, a lack of consensus can be observed among the inquirers in terms of the medium of feedback to be given to students. In this vein, wavering over the efficiency of grammatical feedback on leaners’ writing tasks, Truscott (2007) states that it is even disadvantageous to the learners’ long-term development in writing proficiency. In contrast, a great bulk of research accentuates the crucial role of feedback in the language learning process (Jacobs, Johnston, Santillo, & Wyatt, 1998; & Zhang, 2008). The investigations, in this regard, mostly concentrate on the major types, and the features of the feedback as well as learners’ responses, indicating the essentiality of feedback provision on the part of instructors (Ferris & Roberts, 2001).

This study attempted to examine the effect of paper-based and web-assisted feedback on writing in Iranian EFL learners’ critical thinking. In other words, this study aimed to determine the more contributive feedback type to Iranian EFL learners’ level of critical thinking in academic writing. Besides, it was endeavored to explore the extent of feedback contribution to the five levels of critical thinking.

The subject of criticality or critical thinking has been a significant issue in the realm of EFL teaching and learning. Fahim, Barjesteh, and Vaseghi (2012) claimed that the contribution of critical thinking is crucial for instructors in different contexts of teaching and learning. In this respect, many scholars have investigated this subject, such as Vdovina and Gaibisso (2013), maintaining that critical thinking is pertinent to quality thinking and it assists students to interact with others, gain information, and understand different concepts sophisticatedly. Shirkhani and Fahim (2011) stated that the integration of critical thinking into EFL classes is critical due to the fact that learners become more responsible for their own criticality while monitoring and assessing their own strategies of learning more skillfully.

Diverse investigations have been conducted in terms of the prominence of critical thinking in EFL contexts. For example, Moafian and Ghanizadeh (2011) revealed that there is a significant relationship between EFL learners' critical thinking and their self-efficacy beliefs. However, Zarei and Haghgoo (2012) indicated that the relationship between both vocabulary and critical thinking and between grammar and critical thinking was not statistically meaningful. Besides, Birjandi and Bagherkazemi (2010) state that in order to make learners ready for ways of thinking, teachers themselves need to be able to think in those terms. In this respect, Ketabi, Zabihi, and Ghadiri (2012) illustrated that EFL teachers tend to express strong support for the inclusion of critical thinking into ELT curriculum. So, to clarify this issue, it appears that Ghaemi and Taherian (2011) confirmed the meaningful relationship between EFL teacher's criticality and their teaching attainment. In the same vein, Aliakbari and Sadeghdaghighi (2013) accentuated the requirement to teach learners to ponder critically, and that it has become a major consideration among educators and researchers in recent years.

As practical confirmations to the significance of critical thinking in the realm of teaching, a number of studies revealed the point. For instance, Alizamani, Khodabandehlou, and Mobashernia (2013) provided evidence showing the positive impact of critical thinking competence on reading comprehension skills. Also, in this regard, Khatib and Alizadeh (2012) revealed that the application of literary texts in reading comprehension classes can be regarded as an effective instructional strategy in improving students’ critical thinking skills. Additionally, Bayati and Ghafournia (2016) confirmed the significant relationship between Iranian EFL professors’ critical thinking skills and their professional achievement. Moreover, Clark and Paulsen (2016) asserted that teacher preparation programs should emphasize modeling critical thinking in order for student instructors to integrate and implement problem-solving, evaluating, creating, and many other critical thinking strategies during the student teaching practices.

Another investigated approach to the importance of criticality deals with the advantages of applying social interactions conducive to critical thinking ability. For example, Hajhosseini, Zandi, and Hosseini (2016) confirmed that the subcategories of critical thinking dispositions and social interaction were mostly shown during discussions. There are various ways to provide interactions such as feedback through oral, written, computer-mediated, or web-instigated modes. Sometimes students may not pay attention to comments due to ignoring the purpose of the feedback process. A seemingly reasonable strategy to encourage students to attend to feedback comments is to give a temporary grade, but ask students to talk critically about their work. In fact, students are required to be critically involved in perceiving the crucial goals of feedback (Nicole & Draper, 2008).

A large number of studies have been done on different types of feedback such as Fazio, (2001), Ferris and Roberts (2001), Hyland and Hyland (2001), Min (2006), and Tsui and Ng (2000). In the same vein, Wen's (2013) investigation dealt with the effective role of teacher written feedback on L2 student writings through engaging students in interactions devoid of scores empowering their critical thinking ability as well as their positive emotions. In addition, Srichanyachon (2012) explained about students’ feedback preferences in terms of content, and suggested instructional practices to assist teachers to provide effective written feedback for their students. A reasonable justification that might affect students' tendency and ability to think critically may be the features of feedback. Purnawarman (2011) suggested that providing feedback can be effective in reducing students’ grammatical errors in their writings. This effectiveness can lead to gaining writing quality as a result of their thinking or a type of metalinguistic information obtained about the nature of their errors (Ellis, Loewen, & Erlam, 2006).

Carless (2006) believes that feedback provides students with an opportunity to evaluate their self-image, and think about the required steps to improve their performance. Accordingly, feedback may empower the students to recheck and revise their thinking or attitude towards the given tasks as well as their own performance (Schwartz & White, 2000) so as to match their actual performance with the desired one (Brookhart, 2003). Similarly, Puegphrom and Chiramanee (2011) maintained that feedback entails a self-monitoring thinking process viewing learning as a kind of collaborative, and self-evaluative task. In other words, as Van den Berg, Admiraal and Pilot (2006) explained, feedback may develop a clearer perception of work quality, and thus engage students in an effective self-regulation of their own learning process.

In a similar vein, Yang andWatkins (2010) also confirmed that feedback can be beneficial in developing critical thinking, learner autonomy and social interaction among students. In fact, it induces a dialectic interaction at a thinking level that is informal and potentially more accessible in students' learning process (Rolliston, 2005). It is due to this interactive process that feedback becomes beneficial not only for those who receive it, but also for those who provide it, as it leads to criticality improvement, and simultaneously entails promotion in the ability to self-evaluate objectively according to given criteria (Nicol & MacFarlane-Dick, 2006). Also, the peer assessment as a type of feedback is often challenging for the students, leading to more interaction and expansiveness in practicing peers' recommendations (Strijbos, Narciss, & Dunnebier, 2010). Similarly, Yang, Badger, and Yu (2006) confirmed that peer feedback induces a sort of uncertainty associated with misunderstanding and miscommunication, leading to more thinking, discussion, search for confirmation, and self-correction. Indeed, through peer feedback process, learners can observe other learners' performances, leading to a more dialogic thinking as well as gaining a better understanding of the learning objectives; in other words, it can improve learners' on-task thoughts, and inspire them to think more deeply about the quality of their performance (Nicol & Draper, 2008).

In an empirical research, Sayed (2010) showed that the learners of writing provided with web-based feedback are more successful persuasive writers. Also, Blankenship (2007) stated that the use of blogs in the writing classroom creates a cooperative learning context. Moreover, Ji-jun (2013) illustrated that automated faculty feedback was superior to human faculty feedback. Furthermore, Rostami and Hoveidi (2014) found that students had a positive attitude toward writing on the internet along with their peers, motivating them to provide an interactive environment. Similarly, Godwin-Jones (2006) confirmed that the web-based feedback provides an appropriate space for group learning which is more student-centered, entailing interactions as well as critical analysis.

Particularly, the present research pursued to respond to the following research questions:

• · Is there any difference between the effects of paper-based and web-assisted feedback on Iranian EFL learners’ level of critical thinking in writing?
• · Are there any differences between the five components of critical thinking (inference, assumption, deduction, interpretation, and argumentation), contributed by the feedback types?

Method

Design and Setting

This study is a quantitative research including a non-randomized control group pre/post-test design. This design, in the present study, encompasses two experimental groups receiving paper-based and web-instigated feedbacks along with one control group receiving no feedback. The study was conducted in the department of teaching English as a Foreign Language at the University of Hormozgan context in the city of Bandar Abbas, the Southeastern Iran.

Participants

The total number of 61 participants of the study included three groups of BA students (both male and female) of TEFL at University of Hormozgan, Bandar Abbas, Iran. Their age range was 21 to 25. Each group contained 23 to 25 students. They were all at intermediate level of proficiency based on their scores on OPT test which they took before conducting the writing tasks.

Instruments

The instruments utilized in this study consist of an Oxford Placement Test 2012 developed by Oxford University Press and a test of Critical Thinking, that is, Watson-Glaser Critical Thinking Appraisal (W-GCTA). To evaluate learners' critical thinking ability, the "Watson-Glaser Critical Thinking Appraisal" (CTA) (Online Version) was employed. The online version of Watson-Glaser Critical Thinking Appraisal (W-GCTA) is composed of a set of five tests. Each test is related to somewhat different aspect of critical thinking:

1. Inference. Discriminating among degrees of truth or falsity of inferences drawn from given data.

2. Assumption. Recognizing unstated assumptions or presuppositions in given statements or assertions.

3. Deduction. Determining whether certain conclusions necessarily follow from information in given statements or premises.

4. Interpretation. Weighing evidence and deciding if generalizations or conclusions based on the given data are warranted.

5. Argumentation. Distinguishing between arguments that are strong and relevant and those that are weak or irrelevant to a particular issue. This test involves 86 items and includes 5 subtests, namely drawing inferences, recognizing assumptions, making deductions, interpreting evidence, and evaluating arguments, each comprising 14, 14, 21, 12, and 25 items, respectively. The reliability index of the Critical Thinking Questionnaire regarding the samples of the present study was high (i.e. 0.83) using Cronbach's Alpha.

Data Collection Procedure

Three groups were assigned including two experimental groups and one control group. The three groups were pre/post-tested in terms of their level of critical thinking. Besides, each of the two experimental groups received a particular type of feedback as its treatment
(i.e. paper-based feedback in the first group, and web-assisted feedback in the second group). The control group were taught with no feedback. The data were collected through a period of 11 sessions lasting for 6 weeks. Parenthetically, the medium of instruction and feedback was in English in the three groups.

Data Analysis Procedure

To analyze the data, one-way between-groups ANOVA was applied to explicate the level of divergence between the critical-thinking scores of the three groups on the basis of the type of feedback they received. Additionally, to explore the within-group variation in the level of critical thinking from pretest to posttest, paired samples t-test was employed. Moreover, the eta squared statistic was utilized to illustrate the effect size of feedback for each group on the whole criticality-score variance, representing a reasonable plateau to compare different cases of feedback. Furthermore, MANOVA was employed to explore the extent of feedback contribution to the five levels of critical thinking.

Results

To answer the first research question of the study, one-way between-groups analysis of variance was run whose output is presented in Table 1.

Table 1. One-Way ANOVA

As Table 1 illustrates, there was a statistically significant difference at the p<.05 level in the scores of critical thinking for the three groups. Employing eta squared formula, the effect size was 0.13, which is a rather moderate effect size (Pallant, 2016). Besides, to reveal the between-groups differences, post hoc comparisons of Scheffe test was utilized whose output can be observed in Table 2.

Table 2. Multiple Comparisons of Scheffe Test

Table 2 illuminates that the mean score of paper-based feedback was significantly different from that of the web-assisted type, that is, the web-assisted group significantly outperformed the paper-based group. In addition, it reveals the significant difference between the mean scores of web-assisted and no-feedback groups; that is, the web-assisted group performed significantly better than the group with no feedback. However, the results did not indicate a significant difference between the mean scores of no-feedback and paper-based groups. Consequently, it can be concluded that the web-assisted feedback had a more meaningful contribution to the learners’ level of critical thinking. In order to answer the second research question, MANOVA was carried out whose results are shown in Table 3.

Table 3. Tests of Between-Subjects Effects (MANOVA)

Table 3 reveals that the contribution of feedback on the whole was merely significant to the fifth level of critical thinking (i.e. argument). To specify exactly the type of feedback triggering such meaningful impact on the argument level of critical thinking, post hoc comparisons of Scheffe test were conducted. Table 4 delineates the respective results.

Table 4. Multiple Comparisons of Scheffe Test

Table 4 indicates that whenever the web-assisted feedback was present, the contribution yielded a significant index. In other words, the table shows that the feedback interaction devoid of the web-assisted case, i.e. paper-based and no-feedback interaction, entailed an insignificant contribution to the argument level of critical thinking. Consequently, it can be said that this was the web-assisted feedback which prompted a meaningful impact on the argument aspect of critical thinking.

Discussion

The findings of this study revealed that this was the web-assisted feedback which prompted a meaningful impact on the argument aspect of critical thinking. This issue can be regarded as in line with Mory's (2004) viewpoint considering feedback as a "critical function in knowledge acquisition" (p. 777). The findings, also, confirm Srichanyachon (2012) who accentuates the significant role of teacher feedback in learning, and what educational researchers have addressed about the way to assist students gain critical thinking skills (Browne & Keeley, 2001; Henderson, 2001; Perkins, Jay, & Tishman, 1993; Torff, 2003).Nicol and Macfarlane-Dick (2006) maintain that a typical instance of formative assessment can be conducted through providing feedback in order to empower learners in self-regulatory style of learning. In addition, Hyland (2003) confirms the significance of teacher feedback maintaining that it functions as a guidance for language development, leading to critical self-correction opportunity. In the same vein, the literature on web-assisted feedback have consistently confirmed that this type of feedback provides a quick way of providing indirect instruction which reduces the teacher’s workload, and students produce more output and report being more motivated due to taking advantage of this type of feedback (e.g. Lavolette, Polio, & Kahng, 2015; Rostami & Hoveidi, 2014). For instance, Dunlap (2005) approves of web-assisted feedback as not only does it reduce intrinsic students' negative attitudes towards learning in isolation, but it boosts their learning advancement. Actually, some studies confirm the computer-assisted feedback superiority over paper-based feedback due to some learner-oriented socio-affective reasons (e.g. Matsumura & Hann, 2004; Schultz, 2000).

Additionally, critical thinking has been the focus of a growing body of literature in the realm of language teaching and learning (Raudenbush, Rowan, & Cheong, 1993; Torff & Warburton, 2005; Zohar, Degani, & Vaakin, 2001; Zohar & Dori, 2003). In this respect, the results of the study can be congruent with Brookhart’s (2003) viewpoint, stating that teacher feedback provides learners with information they need, so they can critically recognize where they are in their learning and what to do next, and consequently, they develop a sense of control over their own learning. Also, in line with the results of the present study in terms of the significant effect of web-assisted feedback on learners’ critical thinking, several former studies confirm similar issues (e.g. Shermis & Burstein, 2003; Warschauer & Ware, 2006). Moreover, some investigations indicating the power of web-assisted feedback to enhance learners' sense of criticality can be regarded as another piece of confirmatory evidence in this regard (Gunawardena, Lowe, & Anderson, 1997; Yang, 2002). In addition, the evaluation of learners' criticality in web-assisted contexts from a practical perspective, conducted by Garrison, Anderson and Archer (2001), heralded another confirmation of web-assisted instruction conducive to criticality improvement.

Conclusion

It can be concluded that language learning development might take place when fostering learners’ level of critical thinking would be emphasized through a sort of feedback which is more personalized, dynamic, and challenging as in web-assisted cases. Consequently, teachers can be regarded as mediated mentors in an environment of trust, support, and professional learning promoted by the institutional leadership (Coe et al, 2014) to both generate and gain the knowledge of diverse mediums of feedback. Learners may become more active and more willing in performing the tasks critically when teacher responds to their assignments through feedback. However, learners appear to become substantially critical thinkers due to the web-assisted type of feedback as an alternative to the paper-based type. Instructors should provide learners with enough opportunity to notice the comments in a critical way so as to fully perceive the intentions behind them (Duncan, 2007). Also, feedback should be multi-modally given so as not to be perceived by learners as an indication of right/wrong prescription. In other words, feedback delivery can be performed as an instructional rather than a correctional aspect of the learning process; that is, a learning-centered mode of feedback delivery should be emphasized over a teacher-centered mode in order to scaffold learners’ reflectivity and criticality aspects of their learning process. If learners can be informed through multimodalities of web-assisted feedback scaffoldings, they might find substantial time and space within their learning processes to strategically think about their to-do tasks while choosing a multitude of resources and gradually developing their critical thinking skills. Although learners might have already employed some of these web-assisted multimodalities, they may not always be conscious of such mediated processes like web-assisted feedback triggering potential knowledge of how to think and act critically while accomplishing the L2 tasks.