Native and Non-native Teachers’ Pragmatic Criteria for Rating Request Speech Act: The Case of American and Iranian EFL Teachers

Introduction

Generally, the study of second language learners’ pragmatic ability is called interlanguage pragmatics (ILP). The prevalence of speech act studies in pragmatics is undeniable. Such studies are performed in both acquisitional areas, which deal with EFL learners’ developmental issues, and comparative areas, which are dominantly of cross-cultural studies (Alcon-Soler & Martinez-Flor, 2008). Another categorization associated with pragmatic studies is Leech’s (1983) and Thomas’s (1983) pragmalinguistic and sociopragmatic division of language knowledge in which linguistic and social dimensions of pragmatic knowledge are dealt with respectively. For example, in conceptualizing a speech act, various linguistic recourses that can be employed to produce a speech act are examples of pragmalinguistic understanding, while considering social norms and politeness concerns in a given situation denotes the sociopragmatic knowledge.

Among several speech acts, due to its important role in our lives, request is one of the dominant ones; as Blum-Kulka, House, and Kasper (1989) put it:

By making a request, the speaker infringes on the recipient’s freedom from imposition. The recipient may feel that the request is an intrusion on his/her freedom of action or even a power play. As for the requester, s/he may hesitate to make requests for fear of exposing a need or out of fear of possibly making the recipient lose face. (p. 11)

As it can be clearly realized, request is a critical speech act due to its directive nature, according to Searle’s (1976) taxonomy, and usually leads to face threatening conditions; therefore, it should be exchanged between the interlocutors with great care (Brown & Levinson, 1987; Searle, 1976).

Various studies (i.e., Blum-Kulka, House, & Kasper, 1989; Jalilifar, Hashemian, & Tabatabaee, 2011; Woodfiled, 2008) tried to analyze different aspects of production of this speech act; however, there are few studies about its assessment, specifically ILP rating (i.e. Taguchi, 2011). Therefore, this study aimed to help us understand the ILP rating patterns of native and non-native EFL teachers which have various implications for EFL teachers, materials developers and learners.

Review of the Related Literature

Usually request act occurs when the speaker (requester) expresses his wants to the hearer (requstee) and wants him/her to do something for his benefits (Trosborg, 1995). According to Yule (1996), besides other speech acts, request can be manifested through both off-record as well as on-record means. There are other categorizations proposed for the request speech act; it can be either direct or indirect, both of which can be conceptualized in three ways: (a) direct, (b) conventionally indirect, and (C) non-conventionally indirect (Blum-Kulka et al., 1989). Indirect requests are associated with more polite situations, while direct request forms denote impoliteness (Brown & Levinson, 1987; Leech, 1983). Table 1, based on works of Blum-Kulka and Olshtain (1984), Blum-Kulka (1989), Blum-Kulka, House, and Kasper (1989), Trosborg (1995), and Van Mulken (1996), presents request categorization.

Table 1. Request Categorization

Moreover, it is important to note that indirect and direct forms of request are conceptualized via various means and, despite some universal characteristics, these means are not consistent across various languages (see Blum-Kulka et al., 1989; García, 1993; Takahashi & DuFon, 1989; Walters, 1979; Wierbzicka, 2003). Blum-Kulka and Olshtain (1984) analyzed the similarities and differences across several languages (English, French, Danish, German, Hebrew, Japanese, and Russian). They divided request into different segments such as address terms, head act, and adjunct to head act; moreover, they found the three hypotheses concerning universal features, namely (a) existence of non-central parts such as internal and external modification; (b) choices of direct and indirect requests; and (c) the three degree of indirectness (as mentioned above).

Studies on request speech act are not merely limited to cross-cultural studies; there are also numbers of interlanguage pragmatic studies about EFL learners’ competence in performing this very speech act (see Barron, 2012; Economidou-Kogetsidis, 2011; Kasper & Schmidt, 1996). For example, Woodfiled’s (2008) comparison of Japanese and German EFL learners’ competence with native speakers revealed that like native speakers, EFL learners tended to employ indirect strategies more than direct strategies. But, they used few internal modifications and ample direct strategies in their requests, unlike what native speakers did.

Moreover, various interlanguage pragmatic studies regarding the speech act of request was conducted in Iranian context. Jalilifar, Hashemian, and Tabatabaee (2011) in their study of Iranian EFL learners’ strategies in request speech acts highlighted some habits of EFL learners’ in requesting such as high-level learners’ overuse of indirect form or low-level learners’ overemployment of direct forms.

With the growing role of pragmatic competence in L2 teaching, some insights into pragmatic assessment have been shared and several methods of assessing speech acts, including request, have been recommended. Role plays, discourse completion tasks (DCTs), multiple choice discourse completion tasks (MDCTs), video and picture prompts, and self-assessment are among the testing techniques for pragmatic assessment of the speech acts of request, apology, and refusal which have been developed based on various contextual factors such as Power, Social Distance, and Imposition (Brown, 2001; Brown & Levinson, 1987; Hudson, Detmer, & Brown, 1995). Roever (2005, 2006, 2007, 2008, 2010) developed various pragmatic testing techniques for the mentioned set of speech acts; his methods, unlike the previous testing techniques that merely dealt with politeness, focused on implicatures and routine formulas and were free from bias toward test takers’ L1 backgrounds.

Among assessment considerations, the raters-related concerns are important, since raters’ facets lead to unreliable evaluations and cause (a) the severity or leniency effect, (b) the halo effect, (c) the central tendency effect, (d) inconsistencies, and (e) the test biases (Bachman, 1990; Eckes, 2005; Myford & Wolfe, 2003). One of the interesting areas of study in raters’ issues and behaviors in assessment is the variations and differences between native English speaker (NES) and non-native English speaker (NNES) teachers’ rating of L2 learners. Several studies revealed some contradictory results. For example, Barnwell (1989) found that native raters are more severe, while others such as Fayer and Krasinski (1987) claimed the opposite.

Moreover, as a result of observations of foreign language classes and raters’ behaviors, several assumptions about language teachers’ considerations concerning L2 rating were proposed. For instance, Alcon-Soler and Martinez-Flor (2008) mentioned the mass of studies and teachers’ focus on pragmalinguistic aspect of language, while neglecting or undermining the sociopragmatic features. However, Alemi, Eslami-Rasekh, and Rezanejad (2014) observed EFL teachers’ considerations regarding both pragmalinguistic and sociopragmatic aspects of language.

Regarding the raters’ variation in pragmatic assessment, Taguchi (2011) investigated the Japanese native teachers’ assessment of request and opinion speech acts. Several similarities and differences in their use of pragmatic and social norms in evaluating were found in raters’ evaluation; some of them mainly considered linguistic forms, whereas others focused on non-linguistic factors. Raters were also diverse in their degree of sensitivity to different types of errors. In addition, Youn (2007) analyzed Korean native teachers’ assessment of non-native Korean learners’ pragmatic knowledge which results showed the inconsistent severity in Korean raters’ ratings of the intended speech act. Alemi and Tajeddin’s (2013) study of NES and NNES teachers’ rating of EFL learners’ refusal production revealed some major differences between these two groups of raters. Moreover, in analysis of Iranian EFL learners’ compliments and compliment responses, similar results were discovered (Rezanjed, 2013). However, none of the previous studies on interlanguage pragmatic rating targeted the assessment process of the request speech act by NES and NNES teachers; therefore, the present investigation aimed to do it with addressing the following research questions:

1. What are the criteria used in NES and NNES rating of EFL learners’ production of the speech act of request?

2. Is there any significant difference between NES and NNES raters in rating scores and criteria mentioned in the speech act of request produced by EFL learners?

Methods

Participants

Participants of this study included 50 Iranian NNES and 50 American NES raters for rating the DCTs. The NNES raters were EFL teachers who were teaching English in various language institutes and universities in Iran and the NNES raters were American EFL teachers who were teaching English in different parts of the world. Since accessing to NES rates was nearly impossible in the context of Iran, they were found in LinkedIn which is a professional social network.  Moreover, the raters of the study involved both genders with different teaching backgrounds and experiences. Another group of this study included 12 Iranian EFL learners from one of the language institutes of Tehran who produced answers to the prepared video prompts. The EFL learners were chosen from the upper-intermediate level, since they had to be capable of understanding the request video prompts and producing the intended speech act.

Instrumentation

Video-prompted DCTs were the main instrument of this study. Compared with written methods, Video-prompted DCTs are superior methods of pragmatic production elicitation with various aspects of interlanguage interaction. They include both verbal and non-verbal properties of the conversation and elicit the intended speech act naturally from the EFL learners (Yamashita, 2008). Six employed video prompts of this study were extracted from American movies and TV-series which covered various contextual variables such as different degrees of power, familiarity, and imposition among the interlocutors. Twelve upper-intermediate EFL learners who could comprehend the video prompts and provide a response to each situation were asked to watch the video prompts and respond to each situation. Finally, some of the EFL learners’ responses were selected based on their relation to the purpose of the study and were transcribed with their specific situations similar to completed WDCTs. The final opted responses were typical instances of Iranian EFL learners’ request productions with their specific pragmatic failure and deficiencies. Moreover, a five-point Likert scale and a blank part were placed under each answer for raters to evaluate and mention their criteria.

Data collection procedure

EFL learners were asked to watch 6 request video prompts and respond to each of them. The video prompts situations and the responses were later transcribed by the authors. Afterwards, each situation was accompanied by only one answer from EFL learners. The final completed video-prompted DCTs were distributed among 50 American and 50 Iranian EFL teachers to rate according to their pragmatic appropriateness from 1-5 on Likert scale and to mention their rating criteria in assessment of each answer.

Data analysis

The study was intended to find out the differences existing among NES and NNES raters’ rating criteria in their pragmatic assessments of request speech act; therefore, mixed-method analysis was employed, which consisted of both qualitative and quantitative analysis. During the qualitative phase, raters’ responses were subject to content analysis for illuminating the various criteria that they had mentioned in their comments. Through quantitative analysis, the frequency was calculated for each of the identified criterion in order to identify the major criteria, and then t-test and chi-square of the rating scores and criteria’s frequencies were computed respectively to estimate the differences between NES and NNES raters’ judgments.

Results

Request rating criteria

To address the first research question, native and non-native raters’ comments about EFL learners’ responses to the video-prompted DCT were analyzed carefully and the following nine criteria, including both sociopragmatic and pragmalinguistic aspects, were found to be prominent in raters’ judgments. The raters’ criteria and definition of each criterion followed by native and non-native raters’ comments on EFL learners’ requests are given in Table 2.

As Table 3 shows, there are some differences between NNES and NES raters’ use of the mentioned criteria. The most dominant criterion for NES raters was politeness, while among NNES the major criteria was formality and social status. The occurrence and dominance of the other criteria across the six situations were not totally compatible between the two groups, neither was the length of the raters’ comments. American NES raters employed politeness, query preparatory and softeners, vagueness and explanation, formality and social status, appropriate alternatives, linguistic appropriateness, directness and indirectness, register, and authenticity and cultural anomalies. However, Iranian NNES raters had different patterns of rating; they mentioned formality and social status, politeness, vagueness and explanation, query preparatory and softeners, directness and indirectness, register, appropriate alternatives, linguistic appropriacy and a lower percentage of authenticity and cultural anomalies.

Evidently, the use of query preparatory and softeners in requests seemed important for NES raters; the presence of ‘please’ (or as they say “magic word”) and modal softeners was highly frequent in their comments and examples. However, NNES raters did not consider this factor as much. Another point which needs to be mentioned is NNES raters’ insistence on paying attention to the degree of formality and social status between the interlocutors; the issue of social status of the addressee seemed significant for them, as they frequently mentioned it in their comments.

Table 3. Frequency of Request Criteria Mentioned by NES and NNES Raters

In the next phase, Table 4 presented below displays the two groups’ descriptive statics to identify the variations of rating scores. As it is depicted in Table 4, NES raters awarded scores were more lenient compared to their counterparts in the first, second, third and the last situations. Moreover, the highest mean in both groups of raters belonged to the third situation. The lowest mean among NES raters is for the fourth situation’s mean and among NNES raters it occurred in the sixth situation’s mean. Table 4 also shows the degree of variations in raters’ scores, which implies the higher standard deviations among NNES raters in most of the situations (all the situations except the last one), which proves the lack of convergence in NNES raters.

Table 4. Descriptive Statistics of Ratings by NES and NNES Raters for Request

NES and NNES raters’ different rating scores and criteria use

The second research question was posed to discover the differences between Iranian NNES raters and American NES raters in rating the request. Therefore, an independent samples t-test was run to compare the difference in request scores rating between NES raters and NNES raters. As it is illustrated in Table 5, the results of the t-test indicate that there was a significant difference between the rating scores of American and Iranian raters in rating the EFL learners’ outputs regarding the speech act of request (t (97) = 2.94, p= .004).

Finally, to understand whether there is any significant differences between NES and NNES raters’ criteria employment, a chi-square test was run. As Table 6 and Table 7 indicate, the result of the chi-square (χ² (17) = 210.84, p< .05) proves that there was a significant difference between NES and NNES ratings of the request.

Table 6. Chi-Square of NES and NNES Raters’ Criteria

Table 7. Symmetric Measures Chi-Square of NES and NNES Raters’ Criteria

Discussion

Similar to other types of assessment, pragmatic assessment is influenced by several factors such as test task, rater characteristics, and rating criteria. The present investigation was conducted to explore ILP rating criteria among American and Iranian EFL teachers whilst rating the request speech act. As a result of analyzing the first research question, nine criteria have been extracted from raters’ comments.

The EFL teachers’ criteria included both sociopragmatic and pragmalinguistic categories among both NES and NNES raters. For example, criteria such as formality and social relationship or politeness belong to sociopragmatic category, while linguistic appropriacy or query preparatory and softeners fit into pragmalinguistic aspect of language.  Regardless of what might have been imagined, the majority of chosen criteria belonged to sociopragmatic features. Some of the criteria, such as politeness, explanation, and directness, have been mentioned in previous studies (Alemi, Eslami-Rasekh, & Rezanejad, 2014; Alemi & Tajeddin, 2013), since they are not merely important in producing request speech act and their use is necessary in performing other speech acts as well, unlike, a criterion like use of query preparatory and softeners which has been used for request speech act more often.

Obviously, there were some differences between NES and NNES teachers’ given scores and preferred criteria which might be due to various reasons such as their different cultural and social backgrounds. NNES raters also mentioned fewer criteria compared with NES raters as observed in previous studies (see Alemi & Tajeddin, 2013; Rezanejad, 2013) which could be because of lack of NNES raters’ knowledge. For example, Iranian EFL teachers were inclined to mention one or two criteria in each situation, while American EFL teachers mentioned their criteria more precisely with more detail. Another noticeable discrepancy observed in NES and NNES raters’ rating is about Iranian raters’ failures to draw on softeners and query preparatory; according to Blum-Kulka and Olshtain’s (1984) category, query preparatory which has been mentioned by our NES raters abundantly, is one of the major strategies in requesting.

Moreover, the result of the mean analysis of rating scores indicated that NNES rated the EFL learners’ answers more severely than NES raters, at least in four of the situations. This proves the severity of NNES teachers, which has been claimed previously by Barnwell (1989); however, in Alemi and Tajeddin’s (2013) study, NES and NNES raters’ rating in assessment of refusal speech act proved the opposite. Moreover, it has also been demonstrated that variations of rating among NNES raters based on the calculated SDs of each group in each situation, as presented in Table 2, are more noticeable than NES raters, which proves the profusion of divergence in NNES teachers.

The deficiencies in NNES raters’ evaluation and the significant different between NNES raters and NES raters highlight the inadequacy of some of the NNES teachers’ pragmatic knowledge and the necessity of developing teacher training courses, especially pragmatic training for NNES teachers (see Alemi, 2012; Bardovi-Harlig & Hartford, 1997; Eslami-Rasekh, 2005; Rose, 2005). Of course, it has to be mentioned that such inconsistencies between NES and NNES raters are not merely limited to Iranian teachers, as it has been pointed out by Taguchi (2011).

The importance of teaching pragmatic knowledge to Iranian EFL learners is also quite clear, as most of their teachers are not trained for teaching and rating English pragmatic performance and they cannot transfer this knowledge to their students. In addition, both groups of raters show their sensitivity to pragmatic deficiencies of the EFL learners’ responses rather than their linguistic mistakes. As Eslami-Rasekh and Eslami-Rasekh (2008) claimed in their study, the effect of pragmatic instruction and awareness is undeniable in L2 teaching programs.

Conclusion

This study revealed nine different criteriathat NNES and NES EFL teachers considered in the course of rating DCTs of Iranian EFL learners about the request speech act. The derived criteria were employed in different amounts by the raters, with politeness being the leading criterion for NES raters and formality and social status for NNES raters.

Furthermore, It has been revealed that raters’ use of each criterion were not consistent across different situations, as some criteria were present in all the situations and some of them have not been mentioned at all in some situations. Moreover, the results of the study indicated that there is a significant difference between NES and NNES raters’ patterns of scoring and employing the criteria. This signifies NNES raters’ deficiencies in rating L2 pragmatic output. Furthermore, the importance of both sociopragmatic and pragmalinguistic awareness besides linguistic accuracy in producing pragmatic output became quite obvious, although their perceptions are not consistent between all raters.

Generally, in this study, NNES teachers were more severe in their ratings, as they assigned lower scores to EFL learners’ request productions compared to NES teachers. They also did not mention some of the important criteria mentioned by NES as much; furthermore, they revealed more variations in ratings of DCTs. The inconsistencies observed in ratings of Iranian NNES raters based on the SDs and inappropriacy of EFL learners’ responses prove the need for teaching the L2 pragmatic conventions and norms to both EFL learners and teachers. Appropriate pragmatic teaching courses for EFL teachers who are deprived of authentic material and input could lead to a more convergent and consistent evaluation and teaching procedure and consequently pragmatically knowledgeable language learners.

Furthermore, the present investigation has some implications that prove the necessity of pragmatic knowledge in EFL curricula. According to raters’ scores and observed pragmatic incompetency of EFL learners’ production, it is urgent to teach L2 pragmatic in language classes. Indubitably, teaching pragmatics can be influential when it is supplemented with appropriate textbooks and syllabi which are filled with authentic samples of target language in various situations, so material developers should be cognizant about different social norms between target society and learners’ cultures. As a result, to insert the pragmatic knowledge within L2 educational system in Iran, where English is taught as foreign language and is culturally and socially distinct from the American context, several alterations in educational policies are required.