Help: Use control+F for fast search.
Cite this online source as:
Hamidi, H. (2015). Research in applied linguistics. Retrieved from http://www.iranelt.com/index.php/introduction-to-research-methods.
Research in Applied Linguistics
Last update: 2/10/2017
Assessment in Research
Recently added to the broad filed of research is the concept of assessment. Assessment is a “fact-finding activity that describes conditions that exist at a particular time. No hypotheses are proposed or tested, no variable relationships are examined, and no recommendations for action are suggested” (Best & Kahn, 2006, p. 22). Brown (2004) delineates the main differences between the traditional and the alternative assessment as follows:
Traditional and Alternative Assessment Characteristics
|No.||Traditional Assessment||Alternative Assessment|
|1||One shot, standardized exams,||Continuous long-term assessment|
|2||Untimed, multiple-choice format||Untimed, free- response format|
|3||Decontextualized test items||Contextualized communicative tasks|
|4||Scores suffice for feedback||Individualized feedback and washback|
|5||Norm- referenced scores||Criterion-referenced scores|
|6||Focus on the ''right'' answer||Open-ended, creative answers|
|8||Product oriented||Process oriented|
|9||Non-interactive performance||Interactive performance|
|10||Fosters extrinsic motivation||Fosters intrinsic motivation|
Analysis of Covariance (ANCOVA)
The analysis of covariance permits the experimenter to eliminate initial differences on several variables, including but not limited to the pretest, between the experimental and control groups by statistical methods. The use of pretest mean scores as covariates is considered preferable to the use of gain scores. Gain scores are problematic for two reasons. First, not all individuals have an equal change for an equal gain. A person who scores very low on a pretest has a great deal more room for improvement than one who scores high. For instance, does a person who gains 50 points from a pretest of 20 to a posttest of 70 actually show more improvement than a person who gains just 30 points but goes from 70 to a perfect 100? ANCOVA takes care of this problem statistically. The second problem with gain scores is their lack of reliability (Best & Kahn, 2006, pp. 170-171).
Backward Build-up Assigning
This is a technique introduced by the author (Hamidi, 2015) which refers to a type of assigning in which participants’ scores are arranged descendingly (from the highest to the lowest) in one column. Each group (say 3 groups in this example) is assigned a number; (1) for the control group, (2) for the first experimental group, and (3) for the second experimental group. Then, numbers for each experimental group is arranged in front of the scores column and is repeated backwards (in reverse). The order can be 3-2-1 and 1-2-3 or 1-2-3 and 3-2-1. This technique seems to be better than tossing the coin, since by descending the numbers downwards the we can reduce the probability of aggregation of high or low scores in one group. This technique also helps us have a better chance of forming two equivalent groups. If the groups prove to be non-equivalent at first, we have to run ANVOCA in which we compare the posttest scores by controlling for the pretest scores. This technique can be used either before homogenizing the participants as a selection technique or after homogenizing the participants as an assignment technique. The following table clarifies the point.
A Sample of the Backward Buildup Assigning Table for the Participants of Three Groups
Cochran's and Mantel-Haenszel Statistics
Cochran's and Mantel-Haenszel statistics can be used to test for independence between a dichotomous factor variable and a dichotomous response variable, conditional upon covariate patterns defined by one or more layer (control) variables. Note that while other statistics are computed layer by layer, the Cochran's and Mantel-Haenszel statistics are computed once for all layers (SPSS 22).
Cohen's kappa measures the agreement between the evaluations of two raters when both are rating the same object. A value of 1 indicates perfect agreement; a value of 0 indicates that agreement is no better than chance, and a value of -1 shows perfect disagreement. Kappa is based on a square table in which row and column values represent the same scale. Any cell that has observed values for one variable but not the other is assigned a count of 0. Kappa is not computed if the data storage type (string or numeric) is not the same for the two variables. For string variable, both variables must have the same defined length (SPSS 22).
A measure of association based on chi-square. The value ranges between 0 and 1, with 0 indicating no association between the row and column variables and values close to 1 indicating a high degree of association between the variables. The maximum value possible depends on the number of rows and columns in a table (SPSS 22).
Control (Constant) Variable
This type of research seeks to find the possible relationship between or among different types of variables. In this type research, no treatment is used and usually no variable is manipulated. Below are one example of the research question and one example of the null hypothesis for a correlational study.
Research question: Is there any statistically significant relationship between classroom management and critical thinking of the Iranian EFL teachers?
Null hypothesis: There is no statistically significant relationship between classroom management and critical thinking of the Iranian EFL teachers.
Descriptive Research (quantitative)
This type of research uses quantitative methods to describe what is, describing, recording, analyzing, and interpreting conditions that exist. It involves some type of comparison or contrast and attempts to discover relationships between existing non-manipulated variables. Some form of statistical analysis is used to describe the results of the study (Best & Kahn, 2006).
Effect and role seem to be two synonymous terms which are often used interchangeably in research studies where research questions are formulated and variables are introduced; however, these two terms should be used cautiously. Effect should be used when there is a treatment and we want to find the result of this treatment on a dependent variable. Role should be used when there is no treatment (like the case of a moderator variable). In finding the relationship between two or more variables or in finding the difference between two moderator variables (for example male and female participants), the term “role” should be used instead. The following examples help clarify the point:
Example 1: The effect of teaching read-aloud techniques on the speaking anxiety of Iranian lower-intermediate students.
Example 2: The role of the gender of EFL teachers in the speaking anxiety of Iranian lower-intermediate students.
Effect sizes can be interpreted in terms of the percent of non-overlap of the treated group's scores with those of the untreated group (Cohen, 1988). Effect sizes are also thought of as the average percentile standing of the average treated (experimental) participant relative to the average untreated (control) participant (Becker, 2000). As to the size, Cohen (1988) defined effect sizes as small (d= .2), medium (d= .5), and large (d=.8), stating that there is a certain risk in inherent in offering conventional operational definitions for those terms for use in power analysis in as diverse a field of inquiry as behavioral science.
A measure of association that ranges from 0 to 1, with 0 indicating no association between the row and column variables and values close to 1 indicating a high degree of association. Eta is appropriate for a dependent variable measured on an interval scale (for example, income) and an independent variable with a limited number of categories (for example, gender). Two eta values are computed: one treats the row variable as the interval variable, and the other treats the column variable as the interval variable (SPSS 22).
Evaluation is concerned with the application of its findings and implies some judgment of the effectiveness, social utility, or desirability of a product, process, or program in terms of carefully defined and agreed on objectiveness or values. It may also involve recommendations for action. It is not concerned with generalizations that may be extended to other settings (Best & Kahn, 2006).
This type of research describes what will be when certain variables are carefully controlled or manipulated. The focus is on variable relationships. As defined here, deliberate manipulation is always a part of the experimental method (Best & Kahn, 2006). However, researchers should pay attention that since most of the research studies (mostly classroom research) done in order to find out the possible effect of a treatment (say task-based language teaching) are conducted without the random selection of the participants, they are called quasi-experimental research.
Example: Investigating the effect of task-based language teaching on the speaking skill of Iranian intermediate EFL learners.
Independent variable= task-based language teaching
Dependent variable= speaking skill
Below are one example of the research question and one example of the null hypothesis for an experimental study.
Research question: Does task-based language teaching have any statistically significant effect on the speaking accuracy of Iranian EFL learners?
Null hypothesis: Task-based language teaching does not have any statistically significant effect on the speaking accuracy of Iranian EFL learners.
Gain Scores (Pretest/Posttest)
Gain scores which are also called difference scores or change scores are used in the analysis of pretest-posttest designs. The gain score is calculated by reducing the posttest score from the pretest score. For example if someone’s posttest score on language proficiency test after an intervention was 80 and his pretest was 60, his gain score will be 20 (80-60= 20). Gain score comparison is used as an alternative to the analysis of covariance (ANCOVA) when one or more assumptions of running the ANCOVA have not been met. However, in using gain score comparison there are two main problems. First, as Best and Kahn (2006) assert, not all individuals have an equal chance for equal gain. The second problem with the gain scores is their low reliability.
A symmetric measure of association between two ordinal variables that ranges between -1 and 1. Values close to an absolute value of 1 indicate a strong relationship between the two variables. Values close to 0 indicate little or no relationship. For 2-way tables, zero-order gammas are displayed. For 3-way to n-way tables, conditional gammas are displayed (SPSS 22).
One of the important elements in most research studies which has been shown to affect learning outcomes is gender. Gender is considered a categorical or a nominal variable (male and female). It is also taken as the moderator variable when the experiment is dealing with males and females.
Grounded Theory, based on Strauss and Corbin (1994), is an approach for developing theory that is grounded in data systematically gathered and analyzed.
It refers to the effect of a feature that is not being tested, but can change or influence the results. For instance, a teacher who is rating a student based on “interest in learning English” may give him a higher rating because he is well behaved in class (Richards & Schmidt, 2010).
This type of research describes what was. The process may involve investigating, recording, analyzing, and interpreting the events of the past for the purpose of discovering generalizations that are helpful in understanding the past and the present and, to a limited extent, in anticipating the future (Best & Kahn, 2006).
Example: An investigation into the benefits and drawbacks of the French revolution.
Information and Knowledge
Simply put, information is the raw material, and knowledge is the finished product.
The inter-rater reliability shows the amount of agreement between two or more than two raters on scoring a specific skill. Based on Landis and Koch (1977), values from 0.0 to 0.2 indicate slight agreement, 0.21 to 0.40 indicate fair agreement, 0.41 to 0.60 indicate moderate agreement, 0.61 to 0.80 indicate substantial agreement, and 0.81 to 1.0 indicate almost perfect or perfect agreement.Cohen’s (1960) kappa and related kappa variants are normally used for assessing inter-rater reliability for nominal (categorical) variables. Possible values for kappa statistics range from −1 to 1, where 1 indicates perfect agreement, 0 indicates completely random agreement, and −1 indicates perfect disagreement.
Landis and Koch’s (1977) classification is re-written below:
0.0 to 0.2 indicate slight agreement
0.21 to 0.40 indicate fair agreement
0.41 to 0.60 indicate moderate agreement
0.61 to 0.80 indicate substantial agreement
0.81 to 1.0 indicate almost perfect or perfect agreement
An interval scale is similar to an ordinal scale except that it has the additional quality that the intervals between the points on the scale are equal. For example, the difference between a temperature of 8◦C and 6◦C is the same as the difference between a temperature of 4◦C and 2◦C. However, we cannot say that a temperature of 8◦C is twice as hot as a temperature of 4◦C because as interval scale does not have an absolute zero (Richards & Schmidt, 2010).
Intervening variables are usually neither observable nor measurable, like frustration, intelligence, or noise. Therefore, they are not usually under the focus of the research. See also variable.
Introduction (in writing paper or thesis)
In writing the introduction for research papers, 3 moves should be taken into consideration. According to Swales (1990, 2004) these moves are:
(1)Establish a research territory: show that the general research area is important, central, interesting, problematic, or relevant in some way. Introduce and review items of previous research in the area.
(2)Establishing a niche: indicate a gap in the previous research by raising a question about it, or extending previous knowledge in some way.
(3)Occupying the niche: outline purposes or stating the nature of the present research. Indicate the structure of the research purpose.
A nonparametric measure of correlation for ordinal or ranked variables that take ties into account. The sign of the coefficient indicates the direction of the relationship, and its absolute value indicates the strength, with larger absolute values indicating stronger relationships. Possible values range from -1 to 1, but a value of -1 or +1 can be obtained only from square tables (SPSS 22).
A nonparametric measure of association for ordinal variables that ignores ties. The sign of the coefficient indicates the direction of the relationship, and its absolute value indicates the strength, with larger absolute values indicating stronger relationships. Possible values range from -1 to 1, but a value of -1 or +1 can be obtained only from square tables (SPSS 22).
A measure of association that reflects the proportional reduction in error when values of the independent variable are used to predict values of the dependent variable. A value of 1 means that the independent variable perfectly predicts the dependent variable. A value of 0 means that the independent variable is no help in predicting the dependent variable (SPSS 22).
Levels of heading in paper or thesis
The style of heading recommended by APA consists of five possible formatting arrangements, according to the number of levels of subordination. Each heading level is numbered in parenthesis on the right. Regardless of the number of levels of sub-heading within a section, the heading structure for all sections follows the same top-down progression. Each section starts with the highest level of heading, even if one section may have fewer levels of sub-heading than another section. See the examples below for more clarification.
3. Method (level 1, title case, capitalized word)
3.1 Participants (level 2)
3.2 Instruments and Materials (level 2 again)
3.2.1 Willingness to communicate. (level 3, bold, lower case)
18.104.22.168 Willingness to communicate inside class. (level 4, bold, lower case, italic)
22.214.171.124 Willingness to communicate outside class. (level 4 again, bold,lower case, italic)
2. Review of the Related Literature (level 1, title case)
2.1 Theoretical Background (level 2)
2.1.1 Four language skills. (level 3)
126.96.36.199 Reading. (level 4)
188.8.131.52.1 Intensive reading. (level 5)
184.108.40.206.2 Extensive reading. (level 5 again)
220.127.116.11.3 Reading aloud. (level 5 again)
18.104.22.168 Writing.(level 4)
22.214.171.124 Listening.(level 4)
126.96.36.199 Speaking.(level 4)
2.1.2 Psychological aspects. (level 3)
2.2 Empirical Studies (level 2)
Limitations and Delimitations
Limitations are the ones which are not under our control, like the time and number of students. Delimitations are the ones that we as researchers decide to have, like the level of the learners. Therefore, if we cannot have more than 50 participants, it is a limitation. If we only consider the intermediate level students, it is a delimitation. If the ratio of male to female students is not the same (boys are more than girls or vice-versa), it is a limitation. If we only consider female learners, it is a delimitation.
A nonparametric test for two related dichotomous variables. Tests for changes in responses using the chi-square distribution. Useful for detecting changes in responses due to experimental intervention in "before-and-after" designs. For larger square tables, the McNemar-Bowker test of symmetry is reported (SPSS 22).
Nominal by Interval Association
In finding the correlation between two variables,when one variable is categorical (nominal, dichotomous) and the other one is quantitative (interval, ratio), Eta should be used to report the relationship. The categorical variable must be coded numerically. For more information see Eta.
A nominal or categorical scale is used to assign values to items or individuals that belong to different groups or categories. For example, we may assign the number “1” to all male students and “2” to all female students in a school. But these numbers are arbitrary and interchangeable. Hence, instead of assigning “1” to male students and “2” to female students, we can assign “1” to female students and “2” to male students (Richards & Schmidt, 2010).
See research question.
According to Labov (1970, cited in Ellis, 2008), good data require systematic observation but the act of trying to observe contaminates the data collected. This is referred to as observer paradox.
Operational Definition (of Key Terms)
For the hypothesis to be testable, the variables must be operationally defined. Operational definitions make research methodology more understandable and clearer to the reader by specifying how technical terms are used. Thus, the researcher should specify what “operations are to be conducted, or tests to be used, to measure such variable. Therefore, the hypothesis focuses the investigation on a definite target and determines what observations, or measures, are to be used” (Best & Kahn, 2006, p. 11). In behavioral research many of the qualities or variables of interest are abstractions and cannot be observed directly. “It is necessary to define them in terms of observable acts from which the existence and amount of the variables are inferred. This operational definition tells what the researcher must do to measure the variable” (Best & Kahn, 2006, p. 291).
Example: Motivation is an abstract quality which cannot be observed directly; however, it can be defined operationally as scores achieved on a particular motivation questionnaire having 4 components of intrinsic, extrinsic, instrumental, and integrative motivation. The scores may range ordinally from 1 to 5 in a Likert-sacle format questionnaire where 1 shows the lowest level of motivation and 5 shows the highest level of motivation (ranging from strongly disagree to strongly agree).
Drawbacks: Although operational definitions make research methodology more understandable and clearer to the reader, they have limited meaning since no phenomena can be fully defined. Their “interpretation is somewhat subjective, a fact which may lead experts to disagree about their validity. The fact that numerical data are generated does not ensure valid observation and description, because ambiguities and inconsistencies are often represented quantitatively” (Best & Kahn, 2006, p. 291).
To provide a clear, precise, and concrete definition of a variable (independent or dependent) to be tested in such a way that it can be measured practically. See operational definition.
An ordinal scale makes use of ordinal numbers (e.g. first, second, third). It ranks items or individuals in order on the basis of some criterion. For example, based on scores on a test, test takers may be rank-ordered as first, second, or third in comparison to others who took the same test. However, the difference between the values on the scale is not necessarily the same. Thus, the difference in points between being first or second on a test may not be the same as the difference between being 21st or 22nd (Richards & Schmidt, 2010).
Oxford Placement Test (OPT)
The Oxford Placement Test is primarily used in order to measure and determine the participants’ level of general English language proficiency and ensure their homogeneity. The OPT is often used by ELT researchers as the language proficiency test in which participants scoring one standard deviation above and one standard deviation below the mean are considered homogenized members. This test consists of 60 items in the form of multiple choice questions, and students are supposed to choose the correct answer from among the alternatives. The required time to complete the test is 30 minutes. The reliability of the OPT has been reported by Hamidi (2015) to be .82 using KR-21 formula having seventy students studying New Interchange 3 and .86 using a test-retest method with a 2-week interval having ninety students almost finishing Four Corners 4, both of which show high reliability index.
Phi and Cramer's V
Phi is a chi-square-based measure of association that involves dividing the chi-square statistic by the sample size and taking the square root of the result. Cramer's V is a measure of association based on chi-square (SPSS 22).
Point Biserial Correlation
When one of the variables in the correlation is nominal (e.g., sex, grade), the point biserial correlation is used to determine the relationship between the levels of the nominal variable and the continuous variable (interval, ratio). The nominal variable does not have to be normally distributed. For example, the number of male vs. female students or native vs. non-native students can have quite a skewed distribution (Hatch & Farhady, 1981). Also see Eta.
In experimental research studies, in order to investigate the effect of a specific treatment, we need to have a pretest and a posttest. The pretest is given to the participants before the treatment starts while the posttest is given to them at the end of the treatment. Pay attention that both pretest and posttest should be the same in nature and question, otherwise the result of the research will be under question. For example, a researcher wants to investigate the effect of picture-based teaching on the vocabulary learning of Iranian elementary EFL learners. The researcher should first design a test of vocabulary (say a 50-item multiple choice test). After piloting the test for its reliability and validity, he should give this test as a pretest to the participants once before the treatment starts and once after the treatment sessions are over. The difference between the mean score of the pretest and the posttest helps the researcher statistically find out the possible effect of the treatment (picture-based teaching).
Qualitative Descriptive Research
This type of research uses non-quantitative methods to describe what is. Qualitative descriptive research uses systematic procedures to discover non-quantifiable relationships between existing variables (Best & Kahn, 2006). A qualitative study aims to explore issues, understand phenomena, and provide answers to the questions through analyzing and making sense of the available data. For example, if a language teacher hands out an open-ended questionnaire to the students to collect some reasons behind their speaking problems, he has done a qualitative descriptive research. Information gathered through the qualitative research can further be applied as a basis of a treatment to be used or manipulated in experimental or quasi-experimental research.
Brown (2001) describes questionnaires as written instruments that include statements or questions that participants must answer to by writing responses or selecting from choices supplied by the questionnaire. He distinguishes among open- and closed-response formats, the former referring to interviews that permit participants to answer orally or in writing, the latter referring to those that need them to choose from accessible choices. Brown summarizes many benefits to the closed-response format. First, the closed-response questionnaire produces uniform data in connection with kind and degree of property. Second, interviews of the closed kind are simple to respond and participants are improbable to skip questions. Third, the capabilities to state closed-response answers numerically make the data easier to encode, analyze, and define. This feature of the data makes them seem visual in nature. Finally, the numerical analysis of closed-response data simplifies the testing of reliability and validity measures.
In explaining the benefits of questionnaires, Dornyei (2003) mentions to the great utility of questionnaires related to researcher time, researcher attempt, and financial resources. Another benefit noticed by Dornyei is the flexibility of questionnaires discussing that they can be used in a various set of conditions with a variety of people and subjects.
Despite that, both writers explain the futilities of questionnaires that impose their use. Brown (2001) starts his debate of futilities by referring to one of the basic subjects with closed-response questionnaires: the narrow range of possible responses. The researcher may disregard potentially relevant answers and not contain them between the ranges of probabilities. Closed-response questionnaires attend to be less investigative in nature as they commonly lead to detections the researcher is assuming. The capability to receive unpredicted answers is almost perfectly removed. Finally, obvious, brief closed-response choices are hard to write.
Albeit Dorynei (2003) opposes with those who states that all questionnaire data are invalid and unreliable, he does summary various potential difficulties that could affect on the outcomes from questionnaire data. He enrolls the following futilities: simplicity and superficiality of answers, unreliable and unmotivated respondents, respondent literacy problems, little or no opportunity to correct the respondents’ mistakes, social desirability (or prestige) bias, self-deception, acquiescence bias, halo effect, and fatigue effects.
See descriptive research.
Quasi-experimental Design (of research design)
Hatch and Farhady (1981) assert that quasi-experimental designs are practical compromises between true experimentation and the nature of human language behavior which we wish to investigate. Such designs are susceptible to some of the questions of internal and external validity. However, given the present state of our heart, they are the best alternatives available to us. By using a quasi-experimental design, we control as many variables as we can and also limit the kinds of interpretations we make about cause-effect relationships and hedge the power of our generalization statements (Hatch & Farhady, 1981).
A ratio scale is similar to an interval scale except that it has an absolute zero, which enables us to compare two points on the ratio scale and make a statement such as “this point is three times as high as that point”. A scale for measuring height is an example of a ratio scale. Thus, we can say that a person whose height is 220 cm is twice as tall as a person whose height is 110 cm (Richards & Schmidt, 2010).
Research has been identified as the systematic and objective analysis and recording of controlled observations that may lead to the development of generalizations, principles, or theories, resulting in prediction and possibly ultimate control of events (Best & Kahn, 2006, p. 25).
See research question.
If the research question is a quantitative one seeking an effect or it is a correlational one, we need null or directional hypothesis. If it is a qualitative one, it usually has no hypothesis. Therefore, a thesis might have 3 research questions but only 2 null hypotheses. Below are some examples:
RQ1. Does the implementation of classroom management techniques have any statistically significant effect on the motivation of Iranian EFL learners?
RQ2. Does the implementation of classroom management techniques have any statistically significant effect on the WTC of Iranian EFL learners?
RQ3. What are the Iranian EFL teachers’ opinion about the implementation of classroom management techniques in their classes?
The first two research questions should be answered and analyzed statistically; therefore, they have a corresponding null hypothesis each. However, the third null hypothesis should be answered qualitatively and the researcher needs to interview the participants or give them an open-ended question sheet to collect their opinions. Hence, the third research question does not have any null hypothesis. See examples below.
H01. The implementation of classroom management techniques does not have any statistically significant effect on the motivation of Iranian EFL learners.
H02. The implementation of classroom management techniques does not have any statistically significant effect on the WTC of Iranian EFL learners.
Role (in research questions and variables)
Technically speaking, a sample is prediction for population. Sampling refers to the process of selecting participants or subjects from a population of interest so that by studying the sample, we generalize the results to the population. We have two major types of sampling; probability sampling and non-probability sampling. Each type includes some sub-categories as follows.
- Simple random sampling
- Stratified random sampling
- Systematic random sampling
- Cluster (area) random sampling
- Multi-stage sampling
Accidental or convenience sampling
- Modal instance sampling
- Expert sampling
- Quota sampling
- Heterogeneity sampling
- Snowball sampling
Scale in statistics and testing is the level or type of quantification produced by a measurement. Four different scales exist: (1) nominal or categorical scale, (2) ordinal scale, (3) interval scale, and (4) ratio scale. Scales can be converted into other scales. However, the direction of scale conversion is only one-way (i.e. a ratio scale → an interval scale → an ordinal scale → a nominal scale), not the other way round (Richards & Schmidt, 2010). For more information see each related title.
A measure of association between two ordinal variables that ranges from -1 to 1. Values close to an absolute value of 1 indicate a strong relationship between the two variables, and values close to 0 indicate little or no relationship between the variables. Somers' d is an asymmetric extension of gamma that differs only in the inclusion of the number of pairs not tied on the independent variable. A symmetric version of this statistic is also calculated (SPSS 22).
A term coined by the author (Hamidi, 2015), which refers to the method of dividing the scores into three levels of low, intermediate, and high (small, medium, and large). For example, if we calculate the language proficiency scores out of 100, scores from 1 to 33 are considered low, from 34 to 67 are considered intermediate, and from 68 to 100 are considered high. The same rule is true for Liket-scale questionnaires; if you have a 5-point scale, divide 5 by 3, and then determine the three levels. Therefore, ordinal scores up to 1.69 are considered low, from 1.7 to 3.39 are considered medium, and scores from 3.4 to 5 are considered high. This method is particularly useful when we want to find out the difference between any performance of low-level and high-level students.
A measure of association that indicates the proportional reduction in error when values of one variable are used to predict values of the other variable. For example, a value of 0.83 indicates that knowledge of one variable reduces error in predicting values of the other variable by 83%. The program calculates both symmetric and asymmetric versions of the uncertainty coefficient (SPSS 22).
Variable (Research Variables)
A research variable is a measurable characteristic or entity that varies. It may vary from group to group, person to person, or even within one person or group over time. A research variable can be either a treatment that causes a change in another variable (independent variable) or can be a result of some manipulation (dependent variable). When a researcher works on task-based language teaching with a group of participants (experimental group) but works on nothing special with another group (control group or placebo group) to find out the effect of the treatment on the vocabulary learning of the participants, the independent variable is the task-based language teaching and the dependent variable is the vocabulary learning. Therefore, in research studies, a researcher manipulates an independent variable to determine if it causes a change in the dependent variable. See the example below for more clarification:
Examples: The effect of 1task-based language teaching on 2vocabulary learning of Iranian 3Intermediate EFL learners: A case of 4males and females.
1Independent variable: task-based language teaching
2Dependent variable: vocabulary learning
3Control or constant variable: Iranian Intermediate EFL learners
4Moderator variable: males and females
Intervening variable: noise, frustration, intelligence of the students, etc.
Pay attention that intervening variables are usually neither observable nor measurable, like frustration, intelligence, or noise. Therefore, they are not usually under the focus of the research.
Becker, L.A. (2000).Effect size. Retrieved from http://web.uccs.edu/lbecker/ Psy590/es.htm.
Best, J. W., & Kahn, J. V. (2006).Research in education (10th ed.). Boston: Pearson Education, Inc.
Bontis, N. (1998). Intellectual capital: An exploratory study that develops measures and models. Management Decision, 36 (2), 63-76.
Brown, J.D. (2001).Using surveys in language programs. New York: Cambridge University Press.
Brown, J. D. (2004).Language assessment: Principles and classroom practice. White Plains, NY: Pearson Education.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.
Cohen, J. (1988).Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
Creswell, J. (2008).Educational research: Planning, conducting, and evaluating quantitative and qualitative research. New Jersey: Pearson: Merrill Prentice Hall.
Dornyei, Z. (2003).Questionnaires in second language research: Construction, administration, and processing. Mahwah, N.J.: Lawrence Erlbaum Associates.
Dornyei, Z. (2007).Research methods in applied linguistics. Oxford: Oxford University Press.
Ellis, R. (2008).The study of second language acquisition (2nd ed.). Oxford: Oxford University Press.
Hair, J., Rolph, A., & Tatham, R. (2009).Multivariate data analysis (7th ed.). New York: Macmillan.
Hatch, E., & Farhady, H. (1981).Research design and statistics. Tehran: Rahnama Publications.
Hatch, E., & Lazaraton, A. (1991).Theresearch manual design and statistics for applied linguistics. New York: Newbury House Publications.
IBM Corp. Released 2013. IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174
Lodico, M., Spaulding, D., & Voegtle, K. (2006). Methods in educational research:From theory to practice. San Francisco: Jossey-Bass.
Mackey, A., & Gass, S. M. (2005).Second language research: Methodology and design. New Jersey: Lawrence Erlbaum Associates, Inc.
Richards, J. C., & Schmidt, R. (2010).Longman dictionary of language teaching and applied linguistics. London: Pearson Education.
Salkind, N. J. (2010).Encyclopedia of research design. Retrieved from http://srmo.sagepub.com/view/encyc-of-research-design/n114.xml
Strauss, A., & Corbin, J. (1994). Grounded theory methodology. In N.K. Denzin & Y.S. Lincoln (Eds.), Handbook of qualitative research (pp. 217-285). Thousand Oaks: Sage Publications.
Swales, J. M. (1990).Genre analysis. Cambridge: Cambridge University Press.
Swales, J. M. (2004).Research genres: Explorations and applications. Cambridge: Cambridge University Press.
Trochim, W. M. K. (2006). Research methods knowledge base. Retrieved from http://www.socialresearchmethods.net.