Challenges and Opportunities of Web-based Assessment in EFL Courses as Perceived by Different Stakeholders

Introduction

The early utilization of computers in classroom instruction can be traced back to the 1950s and 1960s. However, with the spread of the positive implications of using computers in pedagogy, some institutions around the globe initiated incorporating computer-assisted instruction in order to improve the quality of teaching and learning (Karber, 2001). This trend opened up a new aspect to education called computer-assisted language learning (CALL) which is defined as “the search for and study of applications of the computer in language teaching and learning” (Levy, 1997, p. 1). Following that, in the 1980s, language pedagogy began to use computers in testing (Brown, 1997). Computer-assisted assessment (CAA) is the umbrella term commonly used to refer to the use of computers in assessment; however, researchers have coined distinctive terms such as 1) Computer-assisted Assessment, Computer-aided Assessment (CAA) or Computer-mediated Assessment (CMA), 2) Computer-based Assessment (CBA), and 3) Web-based assessment (WBA) or Online Assessment to refer to this new trend in testing. As put by Bull and McKenna (2004) and Conole and Warburton (2005), CBA can be categorized into stand-alone software that only needs a single computer, applications that require closed networks/intranet, and programs that are used to deliver items via public networks/internet (web-based assessment). In web-based assessment (WBA) which is defined as the use of the internet in the activity of assessment (Conole & Warburton, 2005), browsers are used in delivering questions, scoring, and reporting the results (Bull & McKenna, 2004).

Tellingly, preparation for an assessment depends on the perceptions about the assessment (before, during, and after the test) and these perceptions can have either negative or positive impacts on teaching and learning (Gielen, Dochy, & Dierick, 2003; Nevo, 2002). A host of empirical studies have focused upon the perceptions of students and teachers as the main stakeholders (e.g., Aydin, 2013; Celik, 2013; Compton, 2009; Freeman, 1997; Ghanbari & Nowroozi, 2021; Kessler, 2007; Levy, 2009; Matsumura & Hann, 2004; Özden, Erturk, & Sanli, 2004; Sagarra & Zapata, 2008). However, assessment in general and online assessment, in particular, involves not only students and teachers but also several other parties including policymakers and legislators, test developers, professional bodies, and other associations who are interested in the use of psychological and educational testing (Bull & McKenna, 2004). As an important stakeholder, a curriculum designer is supposed to interact with several key individuals including test developers, technical support staff, teachers, and educational counselors (Bull & McKenna, 2004). Therefore, their perceptions have a great impact on their educational decisions regarding WBA programs and the effectiveness of this kind of assessment.

Moreover, according to Davies (1993), parents’ perceptions are also worth investigating since awareness of their perceptions can have numerous benefits for parents themselves and the students, teachers, and policymakers. These benefits include parents’ greater appreciation of their vital roles and understanding of their personal efficacy, the increased linkage between parents, teachers and authorities, and provision of support and guidance to the students by their parents. Given that the positive role of parental involvement in education and assessment has been stressed in several studies (e.g., Nair et al., 2014; Roth, 2008; Zhou & Glick, 2005), the current study aimed to contribute to the emerging body of research into this group’s perceptions of WBA which seems to be an under-researched area of inquiry.

On the other hand, although with the spread and growth of technology, computerized education is utilized in courses for almost all age groups, the majority of previous studies centered on adult learners. Conspicuously, how children, teenagers, and young adults perceive utilization of technology in language learning differs from that of adult learners because children and teenagers are more IT-oriented, which may influence their perceptions of computerized education (Arkhipova, Belova, Gavrikova, Lyulyaeva, & Shapiro, 2017). Considering all these points, this study attempted to seek answers to the following research question:

While engaged in WBA, what are the challenges and opportunities as perceived by students, teachers, parents, and curriculum designers?

Background to the Study

Previous research has offered conflicting evidence regarding teachers’ and students’ perceptions toward the use of computers and the internet in education. On the one hand, willingness toward information and communication technologies (ICT) integration in courses was observed. On the other hand, the possibility of successful implementation of CALL was questioned due to issues such as lack of technological knowledge on the part of students and teachers, insufficient resources, training, technical support, anxiety, and technical problems (Aydin, 2013; Celik, 2013; Ghanbari & Nowroozi, 2021; Kessler, 2007; Liu, Lin, & Zhang, 2017; Murday, Ushida, & Ann Chenoweth, 2008; Widiastuti, Mantra, Sukoco, & Santosa, 2021). According to some scholars (Brown, 1997; Bull & McKenna, 2004; Chapelle & Douglas, 2006; Chapelle & Voss, 2016; Conole & Warburton, 2005; Roever, 2001), the following factors can form certain perceptions about WBA:

• Computer, Internet, and Software Literacy
• Computer and Internet Anxiety
• Accessibility, Administration Procedures, and Security
• Content and Design Format of the Questions, Scoring, and Feedback

These factors can contribute to or hinder the utilization of web-based assessment in a language-learning environment and affect how stakeholders perceive WBA.

Computer, Internet, and Software Literacy

Computer literacy is defined by Robinson (2008) as an understanding of the concepts, terminology, and processes related to general computer usage. Internet literacy, as a new form of digital literacy, is the competency to gain, understand, evaluate, and produce information and communication material online (Livingstone, 2009; Livingstone, Bober, & Helsper, 2005). Online teachers and learners should possess the knowledge of using computers and the internet as well as the knowledge of working with web-based applications to make the most out of their experience.

More specifically, lack of sufficient knowledge of computer and internet use on the part of both teachers and students has been considered as one of the obstacles to the successful implementation of CALL in EFL pedagogy (e.g., Alavi, Borzabadi, & Dashtestani, 2016; Celik, 2013; Dashtestani & Hojatpanah, 2020; Kessler, 2007). In the context of Iran, for instance, Dashtestani and Hojatpanah (2020) reported low digital literacy of EFL students in Iran. The researchers distributed a Likert-scale questionnaire among 364 junior high school students and 20 junior high school teachers and conducted semi-structured interviews among 30 junior high school students, 5 junior high school teachers, and 3 educational policymakers in the Ministry of Education in Iran. The purpose of the study was to investigate the perceptions of English teachers and learners regarding students’ computer literacy. The results of the interviews revealed that both the teachers and students regarded the level of computer literacy of the learners high; however, by running Mann–Whitney U test on the quantitative data, the researchers found that in fact, the students’ level of digital literacy was low to moderate. In addition, the results of the interviews with educational policymakers indicated that they did not have clear plans for improving the learners’ digital literacy.

In a similar vein, Alavi et al. (2016) carried out a study on Iranian English for Academic Purposes (EAP) students and teachers’ perceptions of the learners’ computer literacy level. The authors used a 38-item questionnaire with 4-point Likert-scale items and semi-structured interviews to gather data from 641 undergraduate students of civil engineering and 34 EAP teachers in Iran. The results of analysis of the qualitative data and descriptive statistics of the quantitative data revealed that the participants regarded students’ computer literacy as low and inadequate. Both teachers and students asserted the need for training programs for students to promote their digital literacy.

More recently in the Iranian context, employing a qualitative study, Ghanbari and Nowroozi (2021) attempted to find out how 20 Iranian university English language teachers dealt with the challenges of online assessment posed by COVID-19. To this aim, they made use of in-depth semi-structured interviews with the teachers at different times throughout the study. Also, the teachers were asked to provide narrative accounts of how they dealt with the crisis. The findings suggested that after the shift to online assessment, the teachers experienced serious barriers one of which was their digital literacy.

Some other studies referred to teachers’ lack of technological literacy; however, they regarded students technologically competent and literate. In this regard, Oblinger and Oblinger (2005), for example, asserted that the new generation of learners is digitally literate. They mentioned some of the characteristics of such students. For instance, these learners appreciate immediacy in access to and reception of information; they possess multiple media literacy; they tend to express their feelings and opinions using social media with openness and ease. The authors stated that these learners welcome the implementation of technology in their education.

With regard to educators’ computer literacy, in his book, Prensky (2009) used the term “digital immigrants” (p. 2) for teachers who are required to make use of technology in their teaching after a long time of teaching in the traditional environment. He described this group as instructors who are not competent enough to keep up with technological changes and lack the necessary knowledge to implement computerized education. He further stated that educators are not equipped with adequate computer/internet knowledge to meet the needs of learners. He also described today’s students as “digital natives” (p. 1) and believed that the new generation of students are willing to utilize computerized education and they have sufficient computer literacy due to the fact that they were born into the digital world.

Computer and Internet Anxiety

The use of technology and specifically computers brings with itself a variety of affective issues. The utilization of computers in the instruction demands the teacher and learner to keep up with the technological changes, which in itself causes stress and mental insecurity because they are likely accustomed to traditional ways of teaching and assessment. Previous findings regarding computer anxiety can be categorized into two opposing groups: those that revealed the negative impact of computer anxiety on learners and teachers’ perceptions of computerized education and their achievement (Aydin, 2013; Lai, 2010; Matsumura & Hann, 2004), and the ones that reported positive perceptions of computer and internet-mediated classes and productive performance despite the presence of computer anxiety (Amiryousefi, 2016; Lewis & Atzert, 2000).

Regarding the former, Aydin (2013), for instance, tested EFL school teachers’ knowledge about certain computer software and stated that there is a direct relationship between lack of knowledge about computers and lack of computer usage in the Turkish environment. His findings suggested that lack of computer knowledge caused feelings of doubt and emotional uncertainty and lessened the chances of computer use.

With respect to the latter, Amiryousefi (2016) assessed the relationship between computer anxiety and complexity, accuracy, and fluency (CAF) in online L2 writing tasks of EFL students in language learning institutes in Iran. To gather data, the author chose a pretest-posttest design with 70 low-intermediate-level English learners who were randomly assigned to one of two groups of task repetition or procedural repetition. The students in groups were supposed to write narrative descriptions of some photo stories. In addition, a computer anxiety rating scale (CARS) was distributed among participants containing 19 Likert-scale items. By running descriptive statistics, independent-samples t-tests, and Mann-Whitney U tests, the author found out that computer anxiety did not have a statistically significant effect on learners’ improvement of CAF in online L2 writing tasks.

Accessibility, Administration Procedures, and Security

Miller, Linn, and Gronlund (2009) stated that in order to have an effective and productive assessment, test developers must follow a set of carefully planned procedures that start with setting the goals of the test to interpreting the results, and giving feedback. A lot of books and articles have clarified the steps before, during, and after the tests, most of which are applicable to online examinations too. In this area, various studies investigated different aspects of utilizing computerized software tools for the purpose of instruction or assessment. As an example, Otto and Pusak (2009) discussed the features of CALL authoring tools ranging from simple systems (such as simple tools for creating predefined templates for activities) to complex ones (such as advanced tools for making computer-based instructional programs). They also explored the potential of using authoring tools for the production of computer-based instructional programs and combining these creative products to build an online learning environment. The authors pointed out that good CALL authoring tools enable teachers to produce software specific to the needs of language teaching and learning.

In Iran, Yaghoubi, Malek Mohammadi, Iravani, Attaran, and Gheidi (2008) reported the positive perceptions of e-learning in 110 B.S. and M.S. graduate students taking online learning courses via LMS at two public universities of Iran. The authors gathered data through a 5-point Likert-scale questionnaire and ran descriptive and inferential statistics on the received responses. The results revealed that the students mentioned “flexibility in time and place” and “ease and quick share of learning materials” (p. 4) as the two notable advantages of e-learning programs. Issues such as software incompatibility, lack of readiness in faculty members, and slow internet connections were regarded as serious disadvantages.

Harmon and Lambrinos (2008) examined the effect of two modes of assessment on online exams scores using a cheating prediction model. To conduct the research, the authors used proctored and unproctored modes of assessment in the final exams of two undergraduate courses. The exam conditions were similar in every aspect except for the modes of supervision. To find cheating, the authors used a model that predicted exam grades from independent variables of learner characteristics and compared the R-squared statistic for each exam. The results showed that cheating had definitely taken place in uproctored online assessments.

In sum, computerized assessment opportunities and positive properties that were reported in the previous studies were ease of access (Özden et al., 2004), properly set assessment tools and appropriate administration procedures (Caruso & Salaway, 2007; Özden et al., 2004), sufficient technical support and provision of informative tutorials and announcements (Freeman, 1997), user-friendly interfaces (Freeman, 1997; Özden et al., 2004), secure exam conditions (Özden et al., 2004), and provision of proper encouragement (Liu et al., 2017). The negative properties and challenges that were faced in the aforementioned studies were software flaws and server failures (Caruso & Salaway, 2007; Freeman, 1997; Yaghoubi et al., 2008), poor internet connection (Yaghoubi et al., 2008), lack of security (Harmon & Lambrinos, 2008), lack of provision of informative, and sufficient tutorials and technical support (Caruso & Salaway, 2007; Özden et al., 2004).

Content and Design Format of the Questions, Scoring, and Feedback

Some researchers argue that software creators can influence the process of teaching and learning by using certain kinds of questions, providing specific feedback methods, and setting special formulae and criteria for scoring (Bull & McKenna, 2004). In this regard, Özden et al. (2004) asserted the advantage of using objective tests for online assessment. The students reported that the feedback gave them the opportunity to reflect on their progress and the page-by-page questions made them comfortable while answering the questions. However, some students believed that the exam style was inappropriate because the questions were randomly chosen from a pool of items. They suggested that first, the question bank should be categorized and then, the exam questions should be selected randomly from these categories and appear on the screen in the form of ordered categories. Another issue mentioned by the students was that they were not able to see their selections on the completed exam pages and make any changes regarding their previous answers (previous responses were automatically hidden by the system to prevent cheating). Overall, the results showed the students’ satisfaction over the assessment system was due to positive features such as immediate scores and feedback after the exam, randomized order of the questions, and item analysis of the questions.

In another study, Sagarra and Zapata (2008) examined the perceptions of using online software for assessing 245 students’ homework in an L2 Spanish course. The authors designed an online workbook using a course management system called ANGEL (A New Global Environment for Learning) in which various activities including grammar, vocabulary, listening, and reading exercises were put. The students were supposed to do the online exercises as homework during two semesters. The authors collected data through a web-based survey containing categorical and seven-point Likert-scale items, interviews, and two language proficiency assessment tests. The results of running descriptive statistics on the data and factorial ANOVA of the scores of the two tests showed that the learners perceived the implementation of the online workbook positively. They were pleased by the possibility of having several attempts, getting instant feedback specific to each student, being able to work at a leisurely pace, accessing materials without difficulty, and facing user-friendly exercise pages. Moreover, the learners appreciated the positive role of the online workbook in language learning. Despite the students’ positive perceptions, some negative aspects of online exercises were also revealed. For example, they mentioned the time-consuming task of completing the online workbook and the impossibility of seeing the correct answers after the second and final attempt due to software configurations. They also preferred to have direct links to listening audio files on the website instead of having to use the audio CD.

Overall, although students’ perceptions of computerized assessment were reported to be positive due to factors such as auto-corrected quizzes and instant feedback on answers after submission (Bull & McKenna, 2004; Freeman, 1997; Özden et al., 2004; Roever, 2001; Sagarra & Zapata, 2008), page-by-page exams (Özden et al., 2004), access to questions and answers after the exam (Freeman, 1997), and randomized questions (Özden et al., 2004), sometimes they were not pleased by some aspects of the design and content of the questions such as disordered and uncategorized questions and inability to make changes to previously-answered questions (Özden et al., 2004), the mismatch between the content and exam items (Yaghoubi et al., 2008), the impossibility of seeing correct answers after the final attempt, and lack of direct links to listening audio files (Sagarra & Zapata, 2008). Also, while teachers and test-developers perceived this type of assessment positively due to reduced time and ease in designing questions, grading the exams, and reporting feedback (Bull & McKenna, 2004; Freeman, 1997; Roever, 2001), an interesting point observed is that in most of the studies, the researcher or the professional test-developers were involved in designing exam items while the teachers were not.

As the literature suggests, different aspects of WBA have been the subject of inquiry in many studies regarding teachers’ and students’ perceptions. However, few studies have been conducted on how WBA is perceived by parents as guiding contributors (Davies, 1993) and curriculum designers as decision-makers in online test development and application (Bull & McKenna, 2004). Also, the courses that previous studies centered on were mostly non-EFL ones or held at universities with only adult participants (e.g. Aydin, 2013; Freeman, 1997; Ghanbari & Nowroozi, 2021; Kessler, 2007; Liu et al., 2017; Murday et al., 2008; Özden et al., 2004; Sagarra & Zapata, 2008). However, the current study focused on EFL courses for young adult learners at private language institutes. Therefore, drawing upon our previous insights, the current study strived to contribute evidence of the perceptions held by four different yet influential groups of stakeholders regarding WBA.

Method

Design of the Study

Given that this study intended to uncover the way WBA is perceived by the participants, it made use of a basic qualitative research design. To this aim, it attempted to provide “rich descriptive accounts targeted to understanding” (Ary, Jacobs, Irvine, & Walker, 2019, p. 391) WBA from four different groups of stakeholders’ points of view. 

Context and Participants

The first implementation of e-learning in Iran dates back to the establishment of Payam Noor University in 1988. This university was founded in order to facilitate distance learning (Notash & Shahrebabaki, 2015). After a few years, with the spread of technology, some higher education institutes such as Tehran University, Amirkabir University of Technology, Iran University of Science and Technology, Shiraz Virtual University, and some Islamic virtual colleges and centers made efforts to utilize online software for pedagogical purposes (Yaghoubi et al., 2008). In recent years, language institutes have also tried to keep up with the new technological changes in the field of language teaching and learning. More recently, not unlike other parts of the world, the sudden shift from real classes to virtual ones caused by the outbreak of the COVID-19 pandemic has had a far-reaching impact on various aspects of classroom assessment in Iran (Ghanbari & Nowroozi, 2021), thereby making almost all educational contexts resort to online assessment procedures.

The private language institute under investigation (hereafter PLI) is one of the most reputable language institutes with several branches in one of the large cities of Iran. In fact, before the COVID-19 outbreak, it had already taken action to modernize its methods of instruction and assessment by using up-to-date facilities and methods. The participants of this study consisting of 3 curriculum designers, 8 teachers, 9 students, and 7 parents were chosen based on convenience sampling from PLI. The data saturation technique determined the number of the participants in that the data collection phase was ended as soon as no new information was forthcoming; however, given that only 3 people made decisions about the nature and methods of WBA at PLI, the number of curriculum designers was less compared to other groups of the participants. The 9 EFL students were selected from young adult learners. There were also 7 parents whose children had already taken internet-based tests at PLI and 8 teachers of young adult levels with teaching experience between 3 to 14 years. The 3 curriculum designers were PLI Educational Group members chosen by the institute authorities from among skillful and knowledgeable teachers with considerable years of teaching experience. The participants took part in the study voluntarily and were informed about the nature and purpose of the research. The four stakeholder groups had been involved in WBA via Moodle since 2018 and were quite familiar with this method of assessment.

Table 1. Demographic Information of the Participants

Moodle

Modular Object-Oriented Dynamic Learning Environment (Moodle) is a free and open-source LMS. It was created by Martin Dougiamas and released in 2002. The idea of Moodle is based on social constructivist pedagogy, a collaborative approach to learning based on the views of Jerome Bruner, Lev Vygotsky, and Jean Piaget, which focuses on the ways social factors affect learners’ understanding and knowledge (Richardson, 2003). Its developers believe that by creating an environment in which learners collaborate with each other and engage in problem-solving activities, they have given a tool to students to construct their knowledge (Crosslin, 2010). Moodle is used in different forms of online education and assessment. It provides various tools to increase social interaction as well as learning opportunities. In addition, some special features are available for the teacher to assess students’ knowledge and progress. Moodle, as the assessment tool of the WBA program in this research, can provide different types of objective questions including drag and drop, essay, matching, embedded answers (cloze), multiple-choice, select missing words, short-answer, numerical, true/false, third-party question types and so on. Subjective questions should be corrected manually by the teacher due to software limitations. Different modes of assessment are applicable in Moodle. Moreover, it can be used more for low- and medium-stakes testing. High-stakes assessment is not recommended because this type of assessment requires high security and the most accurate testing methods for life-changing decisions (Bull & McKenna, 2004; Chapelle & Douglas, 2006). Both types of summative and formative testing are possible via Moodle. It also enables test developers to embed audio, video, and pictures into test items.

In this study, the website for the online activities at PLI was accessible through a link. The students could use the browsers on a desktop/laptop computer or the Moodle application on a smartphone to enter the website and do the activities. The exam page was accessible to the learners at a specific time on a particular date and unavailable after a few days. Each student could have only one attempt and as soon as a learner entered the exam page and saw the questions, he/she had a limited time to submit the answers. Editing responses was only possible before submission of the answers. The scores and correct answers to the questions were provided immediately after submission.

Instrument

Semi-structured interviews were used to gather the necessary data. The interview questions were adopted from available quantitative questionnaires (Heinssen, Glass, & Knight, 1987; Joyce & Kirakowski, 2013; Papanastasiou & Angeli, 2008; Shin & Son, 2007) and then adapted for the purpose of the study. They included: 1) knowledge of computer software, 2) frequency of software usage for personal purposes, 3) attitudes towards integrating ICT in pedagogy, 4) perceived self-confidence in integrating ICT, 5) frequency of using ICT for instructional purposes in the classroom, and 6) school climate and support. For the interview, the researchers asked questions about how the stakeholders perceived the WBA program at PLI and what challenges and opportunities they had faced during the online assessment. The interviewer focused upon technological literacy, the anxiety related to the use of technology, the administration procedure and its relevant aspects, the design of the questions, and scoring.

In order to gain an understanding of the participants’ perceptions of WBA in general, the researchers asked them some questions including how important using computers in education and work was in their opinion, what the strengths and weaknesses of online assessment were, whether they believed WBA had been effective and useful, and whether they preferred to have an online assessment or paper-and-pencil examination in future.

The interviews were conducted in the participants’ mother tongue (Persian) to be run smoothly and to create a stress-free and comfortable environment. Each interview took between 20 to 40 minutes with the average time of 30 minutes. The students and their parents were asked 16 questions (similar in all groups). In addition to these 16 items, curriculum designers and teachers were asked two extra questions. The interviewer was the second researcher, who was an MA student of TEFL and had been teaching English to different age groups at PLI for 9 years. Therefore, she had both an emic view (due to being an English teacher at the institute) and an etic view (due to her being unaware of the perceptions of the other stakeholder groups). She was trained by the first researcher who was her supervisor on how to establish rapport with the interviewees and conduct the interviews. All interviews were audio-recorded for further analysis with the participants’ consent.

Procedures

After reaching the saturation point, the researchers transcribed the audio-recorded files. Since the interviews were conducted in Persian, to quote and report what the participants had said, the transcribed data were translated into English. The researchers then organized the ideas and identified and extracted recurrent themes in the interviewees’ responses to the interview questions. The consistency of the findings and validity of the inferences were checked using peer debriefing by both researchers independently analyzing parts of the data to see whether there was consensus in their interpretations. Also, some of the translated data along with their analysis and interpretation were randomly given to some participants asking them to check the accuracy and conciseness of the translations and their interpretations. Meanwhile, an attempt was made to resolve potential discrepancies through discussion and negotiation.

Results

In this section, because of space limitations, only some illustrative examples are given. Regarding computer and internet literacy, curriculum designers believed that they were literate to an acceptable level. For instance, C1 claimed, “If I consider my professional community and job requirements, my computer literacy is well suited to my needs. I rate it excellent”. In a similar vein, teachers believed in their adequate knowledge in this regard. As stated by T1, “I know a lot about computers. I am okay with them”. Similarly, students emphasized their satisfactory level of technological knowledge. One learner remarked, “Due to the spread of online education and obligatory online courses at school, my computer and internet knowledge has improved to a satisfactory level”. (S2) However, in contrast to the other stakeholders, some parents did not consider themselves as highly competent computer users. In this regard, one parent said, “What I know about computers and the internet can be rated average” (P1). Overall, however, it could safely be argued that well-suited computer and internet literacy is a perceived opportunity contributing to WBA.

Moreover, curriculum designers’ responses indicated their belief in having sufficient knowledge of creating activities that serve instructional objectives. As an example, C1 reported, “I believe in my capability and knowledge in designing exercises for educational purposes”. In a similar vein, C2 remarked, “I am able to design simple activities pertinent to the needs of learners, but not at a professional level”. Also, almost all teachers expressed that their knowledge was adequate to a level that could enable them to design exercises well suited to their needs using computers and the internet. With regard to that, T1 stated, “I make a lot of activities for my students using computers and the internet”. Similar to that, T3 said, “I have the required knowledge of designing exercises to some extent”. Therefore, another perceived opportunity that would likely make WBA more fruitful could be educators’ competency in the utilization of online educational resources.

In response to the question of how they felt while using computers and the internet, curriculum designers did not express any nervousness. As C1 remarked, “I feel confident because I know how to work with computers and the internet. I do not feel nervous or anxious due to my familiarity with technology”. Teachers also expressed their low anxiety while using computers; for example, T1 asserted, “I have a very good feeling when I use computers and the internet”. Also, the majority of students did not report any signs of nervousness while working with computers and the internet. As S3 stated, “Everything is okay when I work with computers and the internet. I feel good about it”. In accordance with students’ statements, most of the parents referred to feelings of comfort and calmness. Overall, the data indicated that nowadays, almost all stakeholders are experiencing more confidence and less anxiety in using computers and the internet, which could be due to the increase in their computer literacy and repeated exposure to computerized tasks in their daily activities. Therefore, the absence of computer and internet anxiety is currently perceived to be an opportunity that could pave the way for more successful utilization of WBA.

In stating ideas about security, curriculum designers reported high chances of cheating in this type of assessment. “There is no security in this kind of assessment. We cannot make sure whether the test-taker is the learner himself/ herself or not. Also, everyone can take photos of the items and share them with others” (C1). In a similar manner, C2 stated, “I rate the security of the questions zero. There was a high possibility of cheating”. The curriculum designers mentioned the procedures they had planned to reduce the possibility of cheating to the extent possible. In this regard, C2 remarked, “Moodle has options for reduction of cheating such as time limit for answering the questions or provision of a variety of questions. However, since you cannot see the test-taker, someone else can take the test instead of the learner”. Furthermore, the data revealed that teachers emphasized the insecurity of the items and the exams. One teacher stated, “I rate the security of the exams average. The students could open their books and cheat. Even if we put time limits for the quizzes, another person might take the test instead of the student” (T5). Moreover, T7 responded, “Naturally, in this type of online exam, we do not have any proctors. Therefore, chances of cheating are high”. Students too regarded the security of online assessment as low. In response, S2 said, “I think it was possible for the students to cheat. They could take pictures of the items and send them to each other”. Similarly, all parents agreed on the point that cheating could easily occur in these tests. Therefore, lack of security was perceived to be a challenging issue negatively affecting WBA.

In response to the question about the technical staff and office personnel’s support and responsiveness, C2 stated, “The responsiveness was not satisfactory. The students did not know who to contact for their problems about the exams because no professional staff was particularly assigned for that”. He added, “Only the usual office personnel were available. They had to contact different technical support employees to find the answer and report it to the students. Sometimes the technical support staff was not available at the time of the exams”. Accordingly, the majority of the teachers regarded the performance of the personnel in supporting students faulty. For instance, T4 said, “This semester, we did not have a direct operator to answer the questions regarding technical issues. The reports of these problems were sent to several employees in hierarchical ranks and then students were informed of the responses”. Similarly, T5 stated, “Some of my students were not satisfied with the way the technical and office staff responded to them. Even I, myself, received some ambiguous and incomplete answers”. In a similar way, S9 stated, “The office employees could answer our questions only at specific times. If I wanted to take the exam at night, no one was available to solve the possible technical issues and answer me”. Based on these analyses, it can be concluded that insufficient provision of technical support could be perceived as a challenge that could adversely affect perceptions of WBA.

The next question was whether the participants had faced any technical issues while taking the exams and in case of any problems, whether the issue had been solved or not. C2 mentioned some of the technical problems by saying: “Some problems arose while students were taking the exams. Sometimes, the exam webpage did not load for the students and they could not finish the exam in the allocated time”. He added, “Sometimes the students completed the exams and submitted the answers; however, the system did not calculate their grades and they were surprisingly faced with a zero score”. C2 remarked that these problems might come from students’ slow internet connection or Moodle lags and flaws. He asserted, “In my opinion, we were able to solve 50% of the problems. In cases that we were convinced that the students were not responsible for the occurrence of the issue, we allowed them to take the exam again”. In this regard, C3 stated, “We had very few login problems. Most of the issues occurred while taking the exam. For example, the exam page had suddenly disappeared from the screen or in some cases, the exam timer was faulty and showed less exam time than the set one”. He referred to the point that sometimes these issues arose from students’ negligence of the exam tutorials by saying: “Some students did not follow the given instructions and they did not click on the “Finish Attempt” icon. As a result, no score was given to them”. With regard to the attempts in solving these issues, C3 asserted, “If the problems came from server failure or system flaws, we solved them to a great extent. However, if we felt that students’ carelessness was the reason behind these issues, we could not do anything about it”.

Teachers stated that some of their students faced problems, most of which were related to network flaws on the part of the learners themselves. For example, T2 asserted, “One of my students faced internet disconnection and when she was back online, the exam time was over. Of course, the authorities had told the students that they were to make use of a good-quality internet connection”. In this regard, one teacher referred to an interesting feature of Moodle that could help the students when facing internet connection issues by saying: “Moodle is programmed in a way that enables the test-takers to continue taking the exam if they are disconnected or cast out of the system by entering the website again within the set time limit” (T4). One of the students stated, “During the exam, my internet got disconnected several times and my responses were deleted. Surprisingly, when I entered the exam page each time, the countdown timer started from the beginning” (S7). Also, the majority of the parents stated that their children did not have any problems while taking the exams. The gathered data showed facing few technical issues in general. More specifically, there were very few cases of software malfunctions and server failures. Therefore, it could be concluded that few software flaws emerged from the data as a perceived opportunity for successful WBA implementation. Although the reports of facing technical problems were scarce in number, low-speed internet connection was perceived to be the major reason behind most of these issues.

Regarding teachers’ presence during the exams, C1 asserted, “I think teachers’ presence is not needed in online examinations if the instructions are clear and straightforward”. Almost all teachers agreed with curriculum designers; as T1 stated, “I think only the presence of a technical support provider is vital. I do not see any purpose in teachers’ presence at the time of the exams. In my opinion, the questions are clear and do not need any explanation”. She added, “Since the students take the exams at home, nothing can stop them from cheating, even with the presence of a proctor”. Moreover, T6 mentioned, “Regarding the configurations of Moodle, it is impossible for the teacher to attend the exam session”. In contrast to previous groups, the majority of the students preferred teachers’ presence during examinations. As an example, S3 stated, “It would definitely be better if my teacher were present”. Moreover, S7 said, “I think teachers should have more active roles during online examinations”. In a similar way, all parents asserted the helpfulness of teachers’ presence in online assessment. As an example, P6 stated, “In my opinion, if teachers were present at the time of the exams, the students would feel supported”. Considering all these ideas, the need for teachers’ presence during the exam could be coded as a perceived challenge in WBA.

Next, the stakeholders were asked regarding the ways PLI encouraged students to take part in WBA. Curriculum designers mentioned that some actions had been taken in order to motivate students to participate in online activities. For instance, C2 remarked, “We tried to familiarize students with WBA gradually. In our first trials, they were given bonus points for completing the activities and after some time, they were obliged to take the exams to gain 10% of the final score”. However, despite the efforts in giving encouragement, curriculum designers did not find them effective. In this regard, C2 mentioned, “All over the world, it is natural that people would be driven by extrinsic motivation to take part in exams. Our students are not an exception. Only when did we impose an obligation on them, they took the exams”. C3 added, “I think the steps we took to motivate the learners were not totally functional”.

All teachers mentioned that students had no motivation other than getting a good grade and going to the next level by answering exam questions. As an example, T5 mentioned, “When talking about online exams in the class, the technical staff always made mention of the 10% mark distribution. They did not promote any kind of intrinsic motivation in learners”. Another teacher said, “Our institute performed better in this respect compared to other institutes. They gradually familiarized learners with WBA by not setting out an obligation to take the exams at first. Still, I think the encouragement was not sufficient” (T6). In a similar vein, students did not report the provision of any special encouragement from the institute. For instance, S5 stated, “The institute has not been successful in giving us motivation and enthusiasm”. Parents, however, were not aware of the provision of any special encouragement to the students for taking online tests. As an example, P1 said, “I do not know if my child were motivated enough to take the tests”. This lack of provision of intrinsic incentives for learners could in itself be a perceived challenge that might gradually deteriorate students’ interest in attending WBA. 

The next interview item was how the participants evaluated the received feedback and score. C1 regarded the current method of providing feedback after the exam as acceptable by saying: “I think the given feedback is acceptable. In our previous paper-and-pencil type of assessment, the students had no chance of seeing the questions and the answers after the exam. They were only provided with their grades”. C3 also mentioned, “As soon as the students submit their answers, the exam result appears on the screen. This immediate feedback is a good feature of online assessment through Moodle”. Regarding the type of feedback, curriculum designers referred to the requirement for more comprehensive comments on students’ answers. For instance, C1 stated, “If it were possible for someone (either the teacher or an employee of the institute) to analyze the responses and give more detailed feedback, it would be better. The possibility of having that kind of feedback should be assessed”. Moreover, C3 remarked, “Educationally speaking, providing test-takers with only correct and incorrect answers is not sufficient. However, this point requires careful evaluation of students’ responses and this is not possible because of our restrictions at the institute”.

Similar to curriculum designers, teachers were satisfied with the immediacy in giving scores and the ability to see the questions after the exams. Almost all of them approved of the provision of informative comments about students’ mistakes too. In this regard, T1 stated, “Only providing learners with wrong and right answers is not enough. Sometimes they will not learn from their mistakes and go to the next level”. She added, “When we had in-class examinations, we could explain the missed points to the students, but this did not happen in online exams”. T7 said, “It would definitely be better if an explanation were given for the correct answers. This way, the students would know the areas that need study and practice”. Almost all teachers believed that an automated feedback system should be put on the website by the institute to give short comments for each item to the learners. In this regard, T2 remarked, “I myself do not have time for that. However, professional teachers who know the content of the books well and have mastery over the teaching material should provide learners with explanations about the vocabulary or grammatical points mentioned in each question”. Moreover, T5 asserted, “Due to lack of time, teachers cannot provide such feedback, but it would be a great idea if the institute prepared some pre-made explanations on Moodle so that students could know the reason why their responses are wrong or right”.

Accordingly, the majority of the students were pleased by seeing their scores immediately after the test. For example, S7 mentioned, “Provision of immediate scores and wrong and right answers was a good feature of online exams”. Nevertheless, with regard to the type of feedback, most of the students requested more detailed feedback. For instance, S4 said, “When I see that my answer is wrong, I would like to know what the problem has been”. In addition, S6 remarked, “It is a good idea to have comprehensive feedback. It would be better if the website told me the reason behind the correct answer”. Therefore, the provision of immediate feedback and access to items and responses after the exam are perceived to be opportunities that could make WBA advantageous over the traditional kind of assessment. Although almost all stakeholders believed that the provision of more detailed feedback could be more helpful, curriculum designers and teachers pointed out that putting explanations on the website is a time-consuming task. Consequently, despite the insufficiency of the given feedback, still, lack of detailed feedback was perceived to be a challenge influencing the successful implementation of WBA.

To enrich the findings, curriculum designers and teachers were asked two more questions, the first of which was whether designing online tests was a simple or difficult task and the next one was whether PLI teachers were able to make exam items. In this regard, C1 stated, “I think designing online tests is a really costly task if we want to make important decisions about the future educational paths of the learners”. He added, “If teachers are to design tests, they should be professionally trained. In my opinion, not everyone can design online test items”. In addition, C2 asserted, “I think it is not easy to design test items. Test developers need mastery of Moodle and test designing to some extent”. He added, “If our teachers are trained well, I think more than 50% of them are capable of making online test items”. In a similar way, C3 maintained, “Making questions for online assessment requires knowledge of testing and using software such as Moodle. Therefore, designing online tests is not a simple task”. With regard to teachers’ abilities in making tests, he remarked, “Making online tests requires training. In my opinion, not all teachers are capable of designing tests even with training”.

Accordingly, teachers did not consider online test development a simple task; for instance, T1 stated, “Since exam security is low in online assessment, the test developer has to design tests in a way that reduces cheating and this concern makes test developing a complex task”. She added, “Not all teachers can design online tests. Those who have the ability in that field should be trained professionally to make test items”. Moreover, T2 said, “Question making for online assessment is a difficult task. Only teachers with enough experience and specialty can design these tests”. Also, T5 remarked, “Test developing for online assessment is not an easy task at all. I think all teachers have interesting and creative ideas for making these tests. However, not everyone can design online exams”. She added, “I myself can only cooperate with test developers and share my ideas, but I cannot develop test items without the supervision of professional test developers”. As the data revealed, curriculum designers and teachers agreed upon the complexity of designing online tests. Both stakeholder groups believed that online test construction requires special skills in both test development and computer/internet use. Therefore, the complexity of online test development was a perceived challenge to the implementation of WBA. The following table summarizes the challenges and opportunities of WBA perceived by the stakeholders participating in this study.

Table 2. Perceived Challenges and Opportunities of WBA

Discussions

With respect to educators’ literacy in the design and implementation of online educational resources, these findings are in contrast to those of Kessler (2007) in which teachers’ feeling of uncertainty about integrating technology in their classes originated from their lack of computer literacy. While in that study the problem was attributed to the teachers’ lack of exposure to technology through formal means such as special workshops or training courses, in the current study, the teachers and curriculum designers participated in workshops specifically designed for the WBA utilization. Also, they were provided with tutorials and video clips to enhance their computer literacy. Considering students’ digital literacy, the findings of this study are in line with those of Oblinger and Oblinger (2005) and Prensky (2009). These two studies regarded learners as digital natives who have become digitally literate due to extensive exposure to technology. In agreement with these studies, the current students perceived themselves as competent users in utilizing WBA. In contrast to this finding, however, Dashtestani and Hojatpanah (2020) reported low computer literacy of Iranian high school students. This controversy could be justified by the fact that because high school students come from diverse families with different economic statuses, access to high-quality technological devices could be limited for a large number of them. However, the students who take extracurricular courses such as English classes usually come from families that are financially secured, thereby maximizing their chances of exposure to computers and the internet and the development of computer literacy.

Another emerged theme, the absence of computer and internet anxiety, corroborates the findings of Lewis and Atzert (2000) who reported positive perceptions of computer and internet-mediated classes and productive performance in spite of the presence of computer anxiety. In that study, the students were reported to face less and less computer and internet anxiety over time when they were repeatedly required to publish projects on web pages and do online activities and this increased technological literacy lessened their computer anxiety. With respect to tutorials and announcements, in the current study, it was found that the tutorials were perceived to be sufficiently informative and helpful. The students were given ample instructions through pictures and video clips on Moodle, PLI Instagram page, and PLI website, which could contribute to their low levels of anxiety. The same finding, however, contradicts Ghanbari and Nowroozi’s (2021) findings. They found that Iranian university teachers reported high levels of stress because of having to engage in online assessment. However, what is worthy of notice is that they had conducted their study during the COVID-19 pandemic which caused a lot of stress for all stakeholders including teachers. The present study, on the other hand, started before the pandemic and the participants were experiencing WBA as part of their routine teaching-learning activities.

Regarding software and server performance, the participants believed in reduced software flaws which contradicts the reports of Caruso and Salaway (2007) and Freeman (1997). In the former study (2007), the students complained about problems in downloading and uploading files and taking online tests using CMS software. They also mentioned networks being down. Similarly, some participants in the latter study (1997) reported corruption of the database, poor backup processes, server problems, and system crashes as challenges to the utilization of TopClass software. Unlike the findings of these studies, the current participants appreciated the flawless performance of the software and server. Although a few bugs were reported, the overall operation of Moodle was reported to be satisfactory. This could be due to the strong design and infrastructure of Moodle as a powerful LMS that has been put to the test repeatedly all over the world.

Considering the security of the tests, the participants recurrently referred to cheating and insecurity of the exams as a major weakness of WBA, which supports the findings of Ghanbari and Nowroozi (2021) who reported teachers’ concerns over the security of online exams and the problem of cheating during such exams. Harmon and Lambrinos (2008) also found the unproctored mode of assessment had heightened the chances of cheating. Although the curriculum designers had taken steps to alleviate this problem by putting time constraints, cheating was still a source of dissatisfaction over the unproctored online assessment program. In contrast, in Özden et al.’s study (2004), the participants rated the fairness of the exams positively. This was due to factors such as presenting items in randomized order, varied placement of the questions from student to student, and giving the online exams in computer labs with the supervision of proctors, which led to the reduction of cheating to a large extent.

Moreover, the current participants were dissatisfied with the provision of technical support and their responsiveness, which is in line with the assertions of Caruso and Salaway (2007) who reported students’ complaints about the disorientation of customer service. In the current study, it was also revealed that low-speed internet connection was a major cause of occasional technical problems. This finding corroborates that of Yaghoubi et al. (2008) in which the students mentioned the low speed of the internet as a challenge to the process of online examination. As both these studies were conducted in the same context, it could safely be argued that the quality of internet connection, seemingly still a problem, needs improvement and more coherent network policies in Iran.

With regard to the teachers’ presence during WBA, the participants’ opinions were contradictory; while curriculum designers and teachers considered it unnecessary due to the nature of online assessment, students and parents believed that they could benefit from instructors’ support during examinations. The reason might be that PLI has not yet been successful in internalizing the concept of online assessment and that is why students, being afraid of facing ambiguous questions, show traces of preference for traditional types of assessment with the teacher being present during the exam time.

Considering the provision of incentives for learners, the stakeholders expressed dissatisfaction over the lack of sufficient encouragement from the institute authorities. According to them, since taking online exams was obligatory, getting a passing grade was the only motivation they had. In a similar vein, Liu et al. (2017) maintained successful implementation of computerized education requires both external motivation (encouragement from the authorities besides formal requirements) and internal motivation (self-interest in technology and eagerness toward computerized education as a result of awareness of the benefits). In this study, the majority of the students and their parents showed a tendency toward the traditional type of assessment, which could indicate they may not be fully aware of the potential advantages of WBA.

Regarding the type of feedback, the participants expected to receive more comprehensive feedback. According to Bull and McKenna (2004), feedback in computerized assessment is often standardized and automated which has a controlling effect on the learner and may not fully lead to learning. On the other hand, the utilization of rich dialogic feedback can foster learner achievement by signaling the areas that need improvement and engaging the students in self-correction. However, this kind of feedback requires one or more raters that check the answers and give feedback manually or complicated algorithms and specific programs that analyze and compare students’ answers with a model or reference answers (Alrehily, Siddiqui, & Buhari, 2018; Bull & McKenna, 2004). Since using such software requires professional computer programmers and skillful test developers, it is not favored by educators who use objective tests. Considering subjective tests, it is possible for teachers to give comprehensive feedback via annotation tools, e-mail, or other available features.

In this study, a challenge to WBA was found to be the complexity of online test development from both teachers’ and curriculum designers’ points of view, which is in line with what Ghanbari and Nowroozi (2021), Compton (2009), and Chapelle and Voss (2016) found. As for the first study in the context of Iran, the sudden shift to online classes and assessment due to the pandemic did not allow the university to provide its teachers with some training courses. That is why the teachers were not ready to engage in online assessment and thereby found it difficult. Compton (2009) asserted that in order to achieve educational goals, educators should possess technological, pedagogical, and evaluative skills. Moreover, Chapelle and Voss (2016) claimed that professionals with skills in various areas (such as test construction and technology use) are needed for computerized assessment. In contrast to the findings of these studies, Freeman (1997) reported educational staff’s belief in the ease of designing online exams. However, those educators actually referred to other aspects of online test construction such as the omission of printing exam papers, the ease of making changes to questions, and reduced time in producing the exam content. Hence, it can be argued that whereas with respect to the skills needed, designing online tests seems to be complicated, considering putting exam items on the website, online test construction could be perceived as rather straightforward.

Conclusions

The findings of this study could be beneficial to different stakeholders in raising their awareness of other stakeholders’ perceptions toward the effective implementation of WBA. More specifically, by becoming aware of teachers’ and curriculum designers’ perceptions, parents may attempt to scaffold their children in eliminating affective barriers experienced during WBA and motivate them by encouragingly and positively talking them into the importance of utilizing online educational resources. In addition, teachers can set better future course objectives and instructional plans by knowing curriculum designers’ perceptions. Becoming aware of other stakeholders’ perceptions, curriculum designers can attempt to locate areas needing revision and more serious attention, find solutions to the problems, and plan useful, up-to-date, and comprehensive programs to prepare EFL students for sitting online examinations. Also, by investigating the comments received from students, parents, and teachers, they can produce sound and informative tutorials and hold special workshops or prerequisite courses for the parties involved in order to familiarize them with the process of the online assessment.

Furthermore, based on these stakeholders’ perceptions and compared to other software (e.g., Caruso & Salaway, 2007; Freeman, 1997), Moodle seems to offer a more promising outlook for the future of WBA. Nevertheless, regarding improving the security of online tests, it is suggested that Moodle developers consider the potential of adding some new features to this software; for instance, the possibility of displaying test takers’ pictures on the screen while they are taking the test. In its current form, the proctor can only monitor the test takers’ presence by checking their names; however, if, in addition to the names, their pictures are also displayed, the possibility of cheating may noticeably be precluded. This new feature could also allow students to see the teacher, thereby not letting them venture into cheating, on the one hand, and contributing to their feeling supported and secured, on the other. 

Evidently, despite such implications, this study is not void of some limitations. First, only one language institute was under investigation in this study, which was mainly due to ease of access to the stakeholders at PLI and its widespread reputation. Second, while in this study, only objective questions were used in the online assessment program, it needs to be born in mind that the utilization of subjective questions could have affected the perceptions of stakeholders. Moreover, given that young adult learners participated in this study, further research can be done with younger learners and children who are taking online exams. Given that socio-economic differences may affect stakeholders’ perceptions, other studies could be conducted on the perceptions of learners who are taking obligatory online courses and exams at public schools. Future studies in other parts of the world could provide us with a more comprehensive picture of the challenges and opportunities different stakeholders face in different cultural and social contexts with diverse infrastructures and facilities. Also, regarding the factors affecting WBA perceptions, further studies may be done investigating other types of assessment such as subjective, formative, oral, other modes of administration such as proctored, lab-situated, high-stakes, and educational software other than Moodle. Moreover, studies could be conducted examining learners’ cheating strategies during online examinations so that necessary precautions could be taken by other responsible stakeholders. All in all, in spite of the perceived opportunities offered by WBA, it seems it is not considered an effective assessment procedure for making important decisions about test takers’ futures due to its serious challenges. Hence, it can be concluded that WBA has a long way to go before it could completely replace traditional testing methods and future research is necessary to ascertain the effectiveness of such modern testing procedures for high-stakes decision making.