ESL TESTING

ESL TESTING & EVALUATION JOHN NELSON PH.D.

EDUC 636 SPRING, 2002

COURSE OUTLINE

SESSION 1 - January 28/30
    Course Introduction and Planning
    Course Assignments and Texts
    Theory - The Nature of Measurement

SESSION 2 - February 4/6
    Theory - Dimensions of ESL Testing - (Bailey Ch. 1)
    Analysis - The Listening & Speaking Skills
    Reading - "Test Stress; Take Out Your Pencils"

SESSION 3 - February 11/13
    Theory - Validity - (Bailey Ch. 4)
    Short Assignment - Listen. & Speak. Items - (Madsen Chs. 7 & 8)
    Techniques - Nelson's Listening Technique
    Reading - "The Secrets Behind the Scores"

SESSION 4 - February 18/20
    Theory - Referencing (Bailey Ch. 3)
    Analysis - Testing Grammar & Vocabulary
    Techniques - Dictation (Bailey Ch. 2)

SESSION 5 - February 25/27
    Theory - Objective Testing (Bailey Ch. 6)
    Short Assignment - Writing Grammar & Vocabulary items (Madsen Ch. 2 & 3)
    Techniques - Developing Subjective Items

SESSION 6 - March 4/6
    Theory - Subjective Testing (Bailey Ch. 6)
    Analysis - Testing Content vs Testing Language
    Techniques - Developing Subjective Items

SESSION 7 - March 11/13
    Theory - Reliability
    Short Assignment - Writing Content vs Language Items
    Reading - "Demystifying the TOEFL Test"

SESSION 8 - March 18/20
    Theory - Item Analysis (Bailey Ch. 9)
    Technique - Table of Specifications
    TAKE-HOME MID-TERM EXAMINATION to be passed out

-----------------Semester Break March 25-27------------------------

SESSION 9 - April 1/3
    Assignment - Mid-Term Examination Due
   Theory - Statistics I (Bailey Ch. 7)
    Discussion - Mid-Term Examination
    Analysis - Testing Reading
    Technique - CLOZE (Bailey Ch. 5)

SESSION 10 - April 8/10
    Theory - Statistics II
           - Testing and Research (Bailey Ch. 8)
    Short Assignment - Writing a Reading Test (Madsen Ch. 8)

SESSION 11 - April 15/17
    Theory - Standardized Testing & Local Tests
    Analysis - Testing Writing
    Technique - Scoring Writing Prompts
    Reading - "Examined Life"

SESSION 12 - April 22/24
    Theory - Alternative Assessment (Bailey Ch. 13)
    Technique - Portfolio Assessment
    Short Assignment - Writing a Writing Test (Madsen Ch. 6)
    Reading - "Assessing Integrated Language and Content Instruction)

SESSION 13 - April 29/May 1
    Theory - Program Evaluation
    Short Assignment - Testing Language Holistically
    Technique - Developing a Testing Program
    Assignment - Test Development

SESSION 14 - May 6/8
    Theory - Testing Affective Factors
    Course Review
    Course Evaluation

Final Examination - May 13/15

COURSE CONTENT
(Numbers in Parentheses Refer to Class Sessions)

Theory:

1. The Nature of Measurement (1)
2. Dimensions of ESL Testing (2)
3. Validity (3)
4. Referencing (4)
5. Objective Testing (5)
6. Subjective Testing (6)
7. Reliability (7)
8. Item Analysis (8)
9. Statistics (9&10)
10. Testing & Research (10)
11. Standardized & Local Tests (11)
12. Alternative Assessment (12)
13. Program Evaluation (13)
14. Testing Affective Factors (14)

Analyses & Short Assignments:

1. Listening & Speaking Skills (2)
    Writing Listening & Speaking    Items (3)
2. Testing Grammar & Vocab. (4)
    Writing Grammar & Vocabulary Items (5)
3. Content vs. Language Testing (6)
    Writing Content and Language Items (7)
4. Testing Reading (9)
    Writing a Reading Test (10)
5. Testing Writing (11)
    Writing a Writing Test (12)
6. Testing Language Holistically (13)

Techniques:

1. Nelson's Listening Technique (3)
2. Dictation (4)
3. Developing Objective Items (5)
4. Developing Subjective Items (6)
5. Table of Specifications (8)
6. CLOZE (9)
7. Scoring Writing Prompts (11)
8. Portfolio Assessment (12)
9. Developing a Testing Program (13)

Readings:

1. "Test Stress; Take Out Your Pencils" (2)
2. "The Secrets Behind the Scores" (3)
3. "Demystifying the TOEFL Test" (7)
4. "Examined Life" (11)
5. "Assessing Integrated Language and Content Instruction" (12)

Major Assignments & Tests:

1. Course Mid-Term (take-home) given out Mar. 18/20, due April 1/3
2. Test Development due April 29/May 1
3. Final Examination (in-class) May 13/15

COURSE ASSIGNMENTS

1. WEEKLY QUIZZES - After each lecture, you will be given a short quiz based primarily on the material of that class. These are to be done outside of class using any resources you like. Collaboration with other students outside of class is encouraged. There will also be items based on materials covered in the Text book. The quizzes will be collected at the beginning of the following class. Late papers will be accepted only in exceptional circumstances. (15% of the course mark)

2. SHORT ASSIGNMENTS - You will be asked to write 6 short assignments each following an analysis discussed in class. Most assignments ask for you to write test items or apply course material to some situation. Their purpose is to have you focus on what particular skills or knowledge or ability specific test items actually measure. You will be asked to have in mind a particular group of learners for these assignments. Your group should be reflective of ESL/EFL students you have taught or plan to teach. Using your experiences and/or imagination is encouraged. Collaboration on these assignments is also encouraged (10% of the course mark)

3. MID-TERM EXAMINATION - On March 18/20, you will be given a mid-term examination which will be turned in on April 1/3 (there will be no class on March 25/27). The exam will be part of multiple choice and part short answer. Most questions will require you to apply something we have talked about in class to particular situations. One major aspect of the test will be for you to critically evaluate the test itself. (25% of the course mark)

4. TEST DEVELOPMENT - SECOND TERM PROJECT - You will be given a hypothetical instructional situation for which you will be asked to construct an appropriate ESL test. The test can utilize any testing techniques appropriate for the situation, and can contain as many items as you decide but must be scored on the basis of 25 points. This project will be evaluated primarily on the appropriateness of the test to the situation, but in part on the creativity exhibited in the test. The project is due on April 29/May1. (25% of the course grade)

5. FINAL EXAMINATION - On May 13/15, we will have a second examination covering the material presented during the second half of the course. It will be an open book/notebook test with item types similar to the Mid-term Examination. (25% of the course grade)

6. ARTICLES - Five articles will be given out through the course. Each illustrates relevant aspects of testing. They will be discussed in class and questions on the two examinations will be based on them. Collectively they address issues of educational evaluation more general than language measurement.

NOTES ABOUT THE COURSE

Objectives: In general, I hope that you will gain a working understanding of language evaluation, confidence in the discipline of language assessment, and a positive attitude toward measuring student language competence. More specifically:

1. Learn TESTING AND MEASUREMENT THEORY sufficiently to use it confidently to your advantage.

2. Know what relevant and important STATISTICS mean, how to compute them and how to make use of them.

3. Learn various, interesting TESTING TECHNIQUES, new and old.

4. Improve your ability to EVALUATE YOUR STUDENTS' ENGLISH SKILLS including their progress and achievement in those skills.

5. Become familiar with GENERAL ISSUES AND TRENDS in the field of testing and evaluation in the United States.

As a result of this course, I hope that you will develop respect for what testing and evaluation can mean, and how you can use them better in your role as an ESL teacher.

Texts:
1. Kathleen M. Bailey, Learning About Language Assessment. Newbury House, Heinle & Heinle, 1998. I have tried many text books for this course and have yet to find one I like. I hope and think that Ms. Bailey's book may be the one. It covers many of the things I am covering in class, but she wraps her presentations in real stories about real teaching situations which makes the theory more meaningful. I like her presentation very much. I will be asking you to relate her situations to things we have discussed in class. She presents more statistics in a very simple way, than we will be covering in the course, but her book is a good reference book for you if for no other reason.
2. Madsen, Harold, Techniques in Testing, Oxford University Press, 1983. This book presents examples of basic testing techniques directed at all four language skills and components of English. It is not required for the course, but it may be helpful in completing those course assignments dealing with testing the skills. Copies are available in the resource center.

Office Hours:
I will not have office space on campus, but I can be reached on campus 45 minutes before and after class. Otherwise, you can reach me during working hours at 301-445-8450 or at home at 301-593-2894. My fax number is 301-445-8457. My Email addresses are jnelson@pgcps.org. Do not hesitate to contact me with any concerns you may have.

I sincerely hope that you will enjoy this course.