Research result 
해외 논문
-
Tense Situations: Tenses in Contrast and Context.
2024-06-17 15:57
-
The Long-term Effect of Automated Writing Evaluation Feedback on Writing Development. English Teaching
This is a longitudinal case study using a mixed-methods research design to track how two Korean university students improved their English writing competence over one year with the aid of automated writing evaluation (AWE) program, Criterion. The participants wrote essays outside of class every month for one year, submitting first and later second drafts. The participants completed a TOEIC writing test at the beginning and end of the study; students’ reflections on their writing development, obtained through interviews and journal entries, were also examined. A comparison of scores, errors, and quantitative measures of fluency and grammatical complexity indicated writing improvement. Both participants used Criterion feedback effectively to render informed judgments and valid corrections. Essay revision based on Criterion feedback yielded more self-directed learning and greater comfort with writing in content courses. It is suggested that the effect of AWE feedback transfers to long-term improvement. The results p
2024-06-17 15:56
-
The Relationship between Text Cohesion Features and English Proficiency for Korean College Students.
This study investigates the relationship between text cohesion features and English proficiency for Korean college students. That is, it examines how cohesion features can be used to distinguish among essays written by different English proficiency levels. The ICNALE (International Corpus Network of Asian Learners of English) corpus was employed in this study and 600 essays on two prompts (i.e., smoking and part-time jobs) written by Korean students were analyzed. The Tool for the Automatic Analysis of Cohesion (TAACO), the recently-developed program for automatic analysis of cohesion, was employed. Two statistical analyses were performed; a multivariate analysis of variance (MANOVA) and a stepwise discriminant function analysis. Results of the study showed that cohesion feature indices were significantly affected by English proficiency, implying that essays written by Korean students with different English proficiency levels can be differentiated in terms of various cohesion features. Results of a stepwise discriminant function analysis revealed that the best predictor for distinguishing three groups of English proficiency is pronoun density. High–level Korean students produced more cohesive essays than mid- or low-level students in that they used pronouns, overlapping arguments, and lemmas as a way for liking ideas across sentences. However, high–level students underused connectives, compared with low-level students. Implications of this study for English writing pedagogy are also discussed.
2024-06-17 15:55
-
The Process-Oriented ESL Writing Assessment: Promises and Challenges.
This study examines a process-oriented ESL writing assessment called the Computerized Enhanced ESL Placement Test (CEEPT). The CEEPT at the University of Illinois at Urbana-Champaign or its non-computerized alternative (EEPT) have since 2000 offered a daylong process-oriented writing assessment in which test takers are given extended time to plan, produce, and revise an essay. This study examines how 100 English as a Second Language (ESL) students take up the opportunity in this assessment procedure to reflect, interact with others, and revise their essays. Specifically, this study investigates what level of revision test takers focused on as well as the extent to which the quality of written products differed between first and second drafts. Results of this study showed that students produced their final drafts in a more coherent manner with complex sentences, as indicated by increased analytic as well as holistic scores, T-units, and a global level of revision. Extracts from essays whose scores increased, stayed the same, and decreased are presented to give a richer sense of how changes in the measures reflected writers’ revisions. This study thus offers insights into a serious attempt to translate a richer and more complex, process-oriented understanding of writing into a large institutional writing assessment.
2024-06-17 15:54
-
A Comparison of Composing Processes and Written Products in Timed- Essay Tests across Paper-and-Pencil and Computer Modes.
The present study attempted to discover plausible differences in composing processes when English as a Second Language (ESL) students write timed-essays on paper and on the computer. For this study, six Korean students with high proficiency in English were asked to write two essays across modes with two different prompts. Upon completion of the test, participants reported on their composing processes in detail. This study also examined the way in which the quality of the written products differs across paper and computer modes. The purpose of this study was to focus research on composing processes and written products across modes at the level of the individual rather than at that of the group. The results of the study show that individual participants are engaged in different ways and to differing degrees by the writing prompts and by the test-taking modality. While the lack of significant differences for essay scores across modes may be suggestive, the number of subjects is too small for generalization about the relationship between modes of composing and essay scores.
2024-06-17 15:52
-
The Predictive Validity of an ESL Placement Test: A Mixed Methods Approach.
This study investigated the relationships between graduate students' placement test scores for English as a second language (ESL) and three measures of academic performance (grade point average [GPA], faculty evaluations, and student self-assessments). Qualitative and quantitative data collection techniques (archival, questionnaires, and interviews) were used to collect data from 100 students and 55 faculty members. Although nonsignificant correlations were found between test scores and GPA, qualitative findings indicated that English skills are an important factor affecting students' course performance. Additional mixed methods analyses found that variations in students' views of academic success and their relevant background knowledge can help explain the overall insignificant relationship between ESL placement test scores and GPA. This mix of methods thus illuminated particular strands of the complex relationships between English proficiencies and graduate-level academic performance.
2024-06-17 15:50
-
Test Review on the Multi-Media Assisted Test of English Speaking: The SOPI Approach.
The Multimedia Assisted Test of English Speaking was designed to assess global speaking competence of Korean speakers of English at Sookmyung Women's University. The test was developed with the help of the American Council on the Teaching of Foreign Languages for their proficiency guidelines and of the Center for Applied Linguistics for their expertise in computerized assessments. The test roughly follows the structure of the Simulated Oral Proficiency Interview (SOPI); it starts with simple tasks and progresses toward difficult tasks. It is a criterion-referenced test, and test takers' speech samples are scored on the basis of proficiency guidelines. The test has showed positive evidence for authenticity, interactiveness, and practicality. However, empirical studies need to be conducted in areas of reliability, validity, and impact as well as bias. This article also addresses practical implications when considering developing similar oral proficiency assessments—SOPI variants—in other contexts where English is used as a foreign language.
2024-06-17 15:46
국내 논문
-
Lexical Sophistication Features to Distinguish the English Proficiency Level Using a Discriminant Function Analysis
This study explored the lexical sophistication features to distinguish the group membership of English proficiency, using the automatic analysis program of lexical sophistication. A total of 600 essays written by 300 Korean college students were extracted from the ICNALE (International Corpus Network of Asian Learners of English) corpus and a discriminant function analysis was performed using SPSS program. Results showed that the lexical features to distinguish three groups of English proficiency are SUBTLEXUS frequency content words, age of acquisition content words, lexical decision mean reaction time function words, and hypernymy verbs. High–level Korean students used frequent content words from SUBTLEXUS corpus to a lesser degree and produced more sophisticated words that can be learned at a later age and take longer reaction time in lexical decision task, and more concrete verbs.
2024-06-20 12:18
-
The Relationship between English Proficiency and Syntactic Complexity for Korean College Students
This study investigates the relationship between syntactic complexity indices and scores on language use in Jacobs et al.(1981)’ analytic rating scale. Syntactic complexity indices obtained from TAASSC program and 440 essays written by EFL students from the ICNALE corpus were analyzed. Specifically, this study explores the relationship between scores on language use and Lu(2011)’s traditional syntactic complexity indices, phrasal complexity indices, and clausal complexity indices, respectively. Results of the stepwise regression analysis showed that phrasal complexity indices turned out to be the best predictor of scores on language use, although the variance in scores on language use was relatively small, compared with the previous study. Implications of the findings of the current study for writing instruction (i.e., syntactic structures at the phrase level) were also discussed.
2024-06-20 12:13
-
A comparison of noun phrase complexity features between native speakers of English and Korean college students using TAASSC program
This study investigates how syntactic features can be used to distinguish essays written by Korean college students from NS college students. Essays from the ICNALE(International Corpus Network of Asian Learners of English) corpus were analyzed using TAASSC(the Tool for the Automatic Analysis of Syntactic Sophistication and Complexity) program. Two statistical analyses were performed. First, a multivariate analysis of variance(MANOVA) was employed to examine whether there are statistically significant differences in noun phrase complexity features between NS and Korean students. Second, a stepwise discriminant function analysis was performed to classify into two groups(i.e., NS and Korean students) in terms of noun phrase complexity features. Results showed that Korean and NS students significantly differed in terms of various noun phrase complexity indices. The best four predictors for distinguishing Korean students’essays from NS students’ essays were dependents per object of the preposition, relative clause modifier per nominal, prepositions per object of the preposition, and dependents per nominal. It was shown that NS students produced more syntactically complex essays than Korean students by employing phrasal embedding. Implications of this study for English writing instruction are discussed.
2024-06-20 00:48
-
The investigation of students’ satisfaction with English education course using flipped learning
This study introduces the flipped learning model of English education course and investigates students’ perceptions of the efficacy of this model. The participants were university students taking a course called Assessment in English Education during spring 2022. At the end of the semester, participants completed the survey with 17 closed- and open-ended items. Also, students who volunteered for the interview were interviewed individually by a researcher. The results showed that a majority of students perceived the components of flipped learning to be useful. Among them, previewing videotaped lectures was judged the most useful activity, followed by Q & A sessions during the lesson and, finally, group discussion sessions. They also perceived that the flipped learning model of the course contributed to self-directed learning of the subject matter. The study also revealed that students with different TOEIC scores perceived the effectiveness of learning activities differently during a lecture; high-level students tended to perceive learning activities useful in checking up the unclear parts of videotaped lectures, compared with low- or intermediate-level students. The findings can provide practical implications of the flipped learning model of English education course, which also can serve as a valuable reference for English education instructors in similar contexts.
2024-06-20 00:45
-
Analysis of Difficulty Level of the CSAT English 3-Point Question Passages Before and After the Introduction of Criterion-Referenced Assessment.
This study investigates the possibility of a difference in the difficulty of 3-point question passages in the CSAT English exam after the introduction of criterion-referenced assessment. In all, 64 CSAT English reading passages with three-point questions from 2013 to 2023 and 13 metrics of the Coh-Metrix program for text difficulty were examined, including technical indices, vocabulary information, vocabulary diversity, cohesion, syntactic complexity, and standard readability. The results reveal significant differences in word syllable count, Flesch Reading Ease(FRE), and Flesch-Kincaid Grade Level(FKGL) metrics. This suggests that the difficulty of 3-point question passages in the English section of the CSAT has slightly increased after the introduction of criterion-referenced assessment. Furthermore, a comparative analysis was done of the difficulty of passages for three types of question formats continuously featured in the English section of the CSAT before and after the introduction of criterion-referenced assessment. The results show no significant difference in text difficulty for the fill-in-the-blank inference question type. However, the difficulty of passages for grammar judgment questions and choosing the appropriate location for a given sentence in the context of a passage showed a slight increase in text difficulty after the introduction of criterion-referenced assessment. Implications of this study for item writing guidelines for both CSAT and classroom tests are also discussed.
2024-06-19 23:58
-
A Comparison of Students’ Satisfaction with and Perceptions of General English and Practical English Program.
2024-06-17 15:42
-
An Analysis of the Optimal Number of Options in Multiple-Choice Items of the National Assessment of Educational Achievement.
This study investigated the effects of reducing the number of options in multiple-choice items on the psychometric characteristics of the English section of the NAEA (National Assessment of Educational Achievement). The purpose of the NAEA is to measure educational progress and achievement and it is administered to all sixth, ninth, and eleventh grade Korean students annually. The English section for the ninth graders is the focus of this study; it is made up of thirty-four five-option multiple-choice items and six constructed-response items. A five-option, multiple-choice English test was converted to a four- and a three-option version by eliminating the least and the second least frequently chosen options. Item difficulty, item discrimination, guessing, reliability, and information function were computed as a function of the number of options. This study employed the 3-parameter IRT model. Results showed that there were no significant differences among the three means of item difficulty as well as item discrimination indices; however, the inspection of the test information function indicated that the five-option version generally yielded the highest amount of test information function over the ability scale. Results indicate that streamlined three- or four-option versions are not likely to function as well as the five-option version at the test level.
2024-06-17 15:41
-
The Optimal Number of Options Based on the Analysis of Language Domain and Item Characteristics.
2024-06-17 15:40
-
An Exploration into Test-taking Strategies and Item Writing Techniques of the English Section of the NAEA
2024-06-17 15:39
-
Faculty and Students’ Perceptions of English Conversation Courses in the General English Program
2024-06-17 15:39
-
The Validity of the New Item Types of the Listening Section of an English Proficiency Test.
This study investigates the validity of the new item types of the listening section of an English proficiency test. The agenda for developing new item types was stipulated through the needs analysis as well as the critical review of popular large-scale English proficiency tests. The tentative test specifications of the new item types were formulated in a way that the English proficiency test enhances validity as well as authenticity. The test instrument for the pilot test consisted of 20 items and the listening stimuli were presented to test takers only once. Students were asked to fill out the survey right after the test and they were also asked to elaborate on their survey responses through individual interviews. The results of item analysis, responses to survey items, and interviews provided future directions for the listening section as follows. First, picture/graph items are introduced in order to decrease overall difficulty of the listening section. Second, the new item types of the long monologue such as problem-solving and note-taking tasks are introduced in a way that they evaluate practical listening comprehension in authentic context. Third, in the case of dialogue completion items, it is advisable that options are not provided in the test booklet, which in turn contributes to construct validity. Results from the current study will feed back into the reformulation of the test specifications for the listening section of the English proficiency test.
2024-06-17 15:38
-
Evidence Supporting a Validity Argument for an English Listening Comprehension Test: Two Prototyping Studies.
2024-06-17 15:37
-
A Case Study on the Long-term Effect of Automated Writing Evaluation Feedback on Writing Development.
2024-06-17 15:36
-
Students’ Perceptions of the Automated Writing Evaluation Feedback in Writing Courses.
2024-06-17 15:35
-
The Washback Effect of the EBB Rating Scale in Writing Courses
This study investigated the washback effect of the EBB rating scale in writing classes. Specifically, this study explored college students’ perceptions of the EBB rating scale, in comparison with the holistic rating scale. The participants were college students taking the academic writing course and they completed the survey right after evaluating their essays using the holistics rating scale as well as the EBB rating scale. The survey results showed that a majority of students perceived the EBB rating scale useful in checking out the mastery of overall lesson objectives. It was also found that students had different perceptions of usefulness of the EBB rating scale according to different background information such as TOEIC scores and self-assessment of English proficiency. Students with TOEIC scores below 500 thought that using the EBB rating scale made them grasp the overall lesson objectives. Also, students who self-assessed their English proficiency low perceived that the EBB rating scale was useful in diagnosing their own writing weaknesses and comprehending criterial questions. The findings can provide practical implications of the EBB rating scale for writing courses, which also can serve as a valuable reference for course planners in similar contexts.
2024-06-17 15:35
-
Development and Validation of the EBB Rating Scale in College Writing Classes.
2024-06-17 15:34
-
Investigating Rater Effects Using Many-Facet Rasch Measurement: An Application of Myford and Wolfe
Myford & Wolfe’s (2003, 2004) papers suggested how multi-faceted Rasch analysis (using Facets) might be used to illustrate the effectiveness and validity of various statistical indicators in detecting and measuring a number of different rater effects using data from judgement-based contexts. While the arguments presented by Myford and Wolfe were clearly of significance for measurement specialists, this study sets out to explore how practically relevant they might be for classroom teachers as well as test developers. This study explores a dataset containing the ratings by a group of 20 raters of 81 writing scripts from a major international English language examination. The purpose of the study was to see to what extent the statistical indicators that Myford & Wolfe proposed were useful for identifying problematic raters in the real-world data set. To do so, this study compared group- and individual-level indicators of three rater effects (severity/ leniency, randomness, and halo effect) between Myford and Wolfe’s simulated data sets and the real-world data set. Rater behavior was modeled using the Facets program, Version 3.64 (Linacre, 2008). A comparison of results showed that the indicators suggested by Myford & Wolfe proved as effective with real data as they had done with simulated data. The implication of this finding is that the use of MFRM using Facets is likely to offer a valuable statistical tool for the exploration of rater effects not only in large-scale test systems but also in classroom assessments. Findings of this study can provide practical implications for English teachers in middle and high schools who will be conducting a small-scale study of rater effects in performance assessments.
2024-06-17 15:33
-
A Comparison of Cohesion Features between Korean and NS College Students using TAACO Program
2024-06-17 15:33
-
The Effect of Elementary School Students’ Variables on their Performance on English Achievement Tests in Gyeonggi Province
2024-06-17 15:32
-
The Analysis of the Current Status of English Private Tutoring and After-school Programs and the Relationship to English Achievement in Gyeonggi Area
2024-06-17 15:30
-
A Study on Applicability of Assessment Tools for the TOSEL Basic through the Examination of the Achievement Standards in 2015 Revised National Curriculum
2024-06-17 15:28
-
Effect of Automated Writing Evaluation Feedback on Korean University Students’ Revision Behavior
This study investigated the effect of automated writing evaluation (AWE) feedback on Korean university students’ revision behavior. Specifically, it explored the extent to which the quality of essays differed between first drafts and second drafts. Furthermore, the way individual students dealt with incorrect feedback in revising essays was examined. In this study, ETS’ Criterion program, one of the most popular AWE programs, was employed. In cases where incorrect feedback was detected, the first and second drafts were closely compared to find out how students altered the text in response to the incorrect feedback, and this was classified into one of three categories: successful changes, unsuccessful changes, and text deletion. The results showed that students’ second drafts were rated significantly higher than first drafts by two NS raters, which indicates that many students followed the revision suggestions made by the Criterion program. As students’ Criterion scores increased, their ability to identify and make successful changes in response to incorrect feedback improved. The findings of the study can advance our understanding of AWE use in an EFL context and should contribute to broader examination of how Korean university students engage in revision of their essays.
2024-06-17 15:27
-
The Potential of Automated Writing Evaluation Feedback in the Korean University Context
In this study, a popular online writing evaluation service, Criterion, developed by ETS, was integrated into writing instruction in the university context. This study examined the pedagogical effectiveness of Criterion feedback in that context, specifically the extent to which students' writing accuracy increased over the course of two papers and students' perceptions of the benefits of feedback were also examined. The results showed that normalized error frequency significantly decreased from paper 1 to paper 2 overall and in the categories of mechanics and style. Students perceived that Criterion feedback, especially grammar, was very useful in revising essays, and it gave them the opportunity to examine their own writing weaknesses. Although normalized frequency of errors in the category of grammar did not decrease from paper 1 to paper 2, students nevertheless perceived grammar feedback as quite useful, as indicated by coding references to "grammar" in journal entries by NVivo analysis. The findings serve to indicate the potential value of automated writing evaluation use in the EFL context, but need to be supplemented by more research with a larger sample in a similar context.
2024-06-17 15:26
-
The Investigation of the Relationship among Lexical Sophistication, Receptive Knowledge, and Productive Knowledge of Vocabulary
2024-06-17 15:25
-
A Computational Investigation of the Relationship between Lexical Sophistication and Writing Competence using the Automatic Analysis of Lexical Sophistication Program
This study explores the relationship between newly developed indices of lexical sophistication and essay scores written by Asian college students. The study employs the Tool for the Automatic Analysis of Lexical Sophistication(TAALES) program. A total of 440 essays written by EFL students from Korea, China, Japan, Thailand, Taiwan, and Indonesia were selected from the ICNALE(International Corpus Network of Asian Learners of English) corpus. The following two research questions were addressed. First, what is the best lexical sophistication predictor of essay scores? Second, what are differentiating lexical sophistication indices among high-, mid-, and low-level EFL students? Two statistical analyses such as the stepwise regression analysis and the discriminant function analysis were performed using the SPSS version 26.0. Results showed that ‘age of acquisition content words’ turned out the best predictor of essay score, indicating that essays with content words that on average are perceived to be learned later received higher scores. Also, two multiword units such as ‘COCA magazine trigram strength of directional association’ and ‘COCA magazine bigram mutual information’ contributed to distinguishing among EFL students with different English proficiency levels. Therefore, EFL students who produced two-word and three-word phrases with stronger associations were classified as the high proficiency group, hence judged to be more proficient in writing.
2024-06-17 15:22
-
The Relationship between Lexical Sophistication Features and English Proficiency for Korean College Students using TAALES Program
This study investigates the relationship between lexical sophistication features and English proficiency for Korean college students. Essays from the ICNALE(International Corpus Network of Asian Learners of English) corpus were analyzed, using TAALES program. In order to examine whether or not there are statistically significant differences in lexical sophistication features across three groups, MANOVA was conducted. Results showed that the lexical sophistication features were significantly affected by English proficiency level. Essays written by Korean students with different English proficiency levels can be differentiated in terms of various lexical sophistication features including content words frequency, content words familiarity, lexical decision mean reaction time function words, hypernymy verbs, word naming response time function words, age of acquisition content words.
2024-06-17 15:21
-
The Relationship between English Proficiency and Syntactic Complexity for Korean College Students
This study investigates the relationship between syntactic complexity and English proficiency for Korean college students, using the recently developed TAASSC(the Tool for the Automatic Analysis of Syntactic Sophistication and Complexity) program. Essays on the ICNALE(International Corpus Network of Asian Learners of English) corpus were employed and phrasal complexity indices and clausal complexity indices, respectively were used to predict English proficiency level for Korean students. Results of stepwise regression analysis showed that indices of phrasal complexity explained 8% of variance in English proficiency, while indices of clausal complexity accounted for approximately 11%. That is, indices of clausal complexity were slightly better predictors of English proficiency than indices of phrasal complexity, which contradicts Biber et at.(2011)’s claim that phrasal complexity is the hallmark of writing development.
2024-06-17 15:20
-
Investigation into Longitudinal Writing Development Using Linear Mixed Effects Model.
This study investigates longitudinal writing development in terms of syntactic complexity using linear mixed effects (LME) model. This study employs essays written by four case study participants. Participants voluntarily wrote essays outside of the classroom and submitted the first and second drafts, after reflecting on the automated writing evaluation feedback (i.e., Criterion) every month over one year. A total of 48 first drafts were analyzed and syntactic complexity features were selected from Syntactic Complexity Analyzer. Results of LME showed that there was a significant positive linear relationship between time and mean length of T-unit and also between time and the ratio of dependent clauses to independent clauses, indicating that case study participants wrote longer T-units and also a higher proportion of dependent clauses over one year.
2024-06-17 15:19
Research
우리 연구실의 연구정보를 안내합니다.
자세히 보기