COURSE CONTENT AND OBJECTIVES

This course elaborates and builds upon the basic skills gained in introductory social science research methods and statistics courses. The conceptual, administrative and methodological aspects of advanced research techniques are addressed. Approaches to evaluating social programs and policies are also covered. Priority is given to the development and utilization of practical knowledge about methodological issues encountered in the research setting. In both the lecture and discussion segments, the course prepares students to assess the quality of research studies as well as to design their own research projects. The readings, lectures and exercises frequently challenge conventional notions of "science" and the "scientific method," and destabilize existing conceptions of "revealing the truth," "objectivity of the researcher," and "value-free research." Hopefully students will improve their analytical skills by examining implicit assumptions and by identifying the costs and benefits of alternative research practices. Specifically, students should learn to:

► understand the political relationships among research methodology, evaluation, and social policy.

► conceptualize research questions, develop appropriate methods, and use appropriate statistics.

► analyze, compare, synthesize, and critique the methodology of research studies.

► design (and know how to execute) manageable research projects.

COURSE PREREQUISITES

This course is an advanced research methodology course. Basic knowledge of the social research process, such as that acquired in SOCY/PUBL 600, is required. Furthermore, familiarity with fundamental statistical procedures, such as that acquired in SOCY/PUBL 604, is assumed. This general knowledge is built upon by elaborating more specific methodological and statistical techniques used in social science research. Please consult with the professor if you are in doubt about your level of preparation for the course.

COURSE REQUIREMENTS

Readings

Required Texts: Weiss, Carol. 1998. Evaluation. Methods for Studying Programs and Policies. Upper Saddle River, NJ:Prentice Hall. [Weiss]

Meenaghan, Thomas, Keith Kilty and John McNutt. 2004. Policy Analysis and Practice. Chicago: Lyceum. [Meenaghan et al.]

Allison, Paul D. 1999. Multiple Regression. Thousand Oaks, CA: Pine Forge. [Allison]

Additional Readings: Additional required readings are on ELECTRONIC RESERVE and some are on BLACKBOARD (BB)

Assignments

Readings, Class Participation, and Attendance. This course will be taught as a seminar supplemented by lectures and hands-on exercises, the success of which greatly depends on the discussion of readings and assignments. Because this course is taught as a seminar, students should attend class regularly, inform the instructor of scheduling problems, and be prepared to summarize and critique the assigned readings. The instructor reserves the right to ask students specific questions about the assigned materials. In order to ensure everyone's motivation for class participation, all class participants will prepare reading reactions for specific assigned readings. You should prepare these reactions by answering the following questions (TYPED):

(1) WHAT was the most important or interesting sentence/statement/argument in the reading and WHY do you think this is the case? Include author and page number.

(2) WHICH sentence/statement/argument in the reading do like least or disagree with and WHY? Include author and page number.

These reactions will form the basis for in-class discussion and will be collected each class period. They vary in points for a cumulative semester maximum of 85 possible points. Because we will discuss specific contents of the readings, it is necessary to bring the assigned readings and notes to class. You should print all required articles. Students are required to use Blackboard (BB). Make sure your UMBC email account is working and you check your e-mail and BB periodically for posted messages and announcements regarding assignments and schedule changes.

Exercises and Written Assignments. There will be several in-class and/or homework exercises. These consist primarily of research problems and students are required to complete and discuss the exercises based on their reading assignments. In order to give you practice in designing various parts of a research project, you will also turn in three (3) written assignments on the dates indicated on the schedule. Assignment 1 requires you to design models for a research proposal (75 points), Assignment 2 deals with table construction based on computer output (50 points), and Assignment 3 involves writing up the results and conclusion sections of an article selected by the professor (75 points). All assignments will undergo a peer review process, in which students will read, and make written comments on, each other's work. This serves to improve your work and give you an idea of your classmates’ writing.

Evaluation or Academic Research Proposal (required for SOCY MA Students only):

Sociology students can either complete an evaluation research proposal, or an academic research proposal. The first option is recommended for students working on a thesis involving the evaluation of a specific program. The second option is best for students whose research interests are geared towards academic rather than evaluation research. In either case, on the date indicated on the schedule, you are required to turn in a one-page outline of your research proposal, containing the research question, the suggested research design and data sources, and a tentative reference list. After getting your research idea approved, you will turn in the final proposal (be sure you have a copy for yourself) at the end of the semester. I expect the papers to be of professional quality, i.e. to be original papers with sole authorship of the course participants. They should be non-published, grammatically correct, contain American Sociological Review or similar citation format, employ non-sexist language, and address the issues clearly and succinctly. Details on format and content are on BB. This requirement includes an in-class presentation as practice for presenting at professional conferences. On the date agreed upon with the professor, you will present your proposal in class. You are expected to prepare an outline with overheads and handouts for the class participants. You will have 15 minutes for the presentation and audience questions and discussion.

Comprehensive Take-Home Final Exam option (required for non- SOCY MA students only)

Students not in the SOCY MA program complete a take-home final. You will receive the exam 2 weeks before the end of classes, have a week to answer the questions, and turn in the exam on the last day of class. The exam resembles a PhD field qualifying exam. In your answers you are expected to integrate substantive knowledge gained from class readings (cited) and apply your knowledge to a research problem.

POLICIES

General

Late work is strongly discouraged and will only be accepted without point deduction in cases of verifiably legitimate reasons (proof of illness or other severe circumstances) for the delay. It is the responsibility of the student to prepare assigned materials on time and to discuss problems with deadlines with the professor. Deadlines missed without promptly (contact the professor on the same day as the missed deadline) presenting a valid excuse to the professor, will be counted as "0" points for the scheduled assignments (no exceptions). Excessive absences or tardiness will affect your final grade negatively. You are responsible for being informed about any scheduling changes or announcements about assignments made in class, via e-mail, or on BB. It is your responsibility to communicate with the professor in person or via e-mail (the telephone is not a good way to get a timely response). Plan ahead in completing assignments. Under exceptional time pressures it is always better to at least "skim" the readings and to attend class than to read nothing and skip class. For your own protection, photocopy all assignments before turning them in.

The nature of the seminar places everyone in a somewhat unfamiliar environment, which does not have to be stressful. I expect students to create a comfortable class environment by sharing ideas, observations, and questions in a respectful manner. Nevertheless, please also refrain from monopolizing the discussion or carrying on disruptive private conversation in class. I encourage you to stop by during my office hours or to make an appointment to discuss your progress or that of the class. If, at any time during the semester you encounter difficulties or special circumstances, contact the professor or the T.A. immediately in order to work out solutions as soon as possible. Incompletes are only given in very special circumstances.

Academic Misconduct

By enrolling in this course, each student assumes the responsibilities of an active participant in UMBC’s scholarly community in which everyone’s academic work and behavior are held to the highest standards of honesty.  Cheating, fabrication, plagiarism, and helping others to commit these acts are all forms of academic dishonesty, and they are wrong.  Academic misconduct can result in disciplinary action that may include, but is not limited to, suspension or dismissal.  To read the full Student Academic Conduct Policy, consult the UMBC Student Handbook, the Faculty Handbook or the UMBC Policies section of the UMBC Directory.  Violations of these rules will result in ZERO points for the assignment and may result in course failure as well as sanctioning by the university.

Academic misconduct may include but is not limited to the following (adapted from UMBC’s policy):

Cheating: Using or attempting to use unauthorized material, information, or study aids in any academic exercise, such as exams, quizzes, papers, homework, etc. That includes copying information from classmates’ papers, homework, exams, quizzes, etc.
Fabrication: Intentional and unauthorized falsification or invention of any information or citation in an academic exercise, such as exams, quizzes, papers, homework, etc.
Facilitating academic dishonesty: Intentionally helping or attempting to help another commit an act of academic dishonesty.
Plagiarism: Representing the words or ideas of another as one’s own in any academic exercise. That means that ANYTIME you use the exact words from a published source (including the internet), you have to put them into quotation marks and attach in parentheses the page number and all biographical source information. If you use the contents of any published source (including the internet) by PARAPHRASING it, you still have to acknowledge it in parentheses.
Buying papers on the internet or elsewhere and turning them, or any parts of them, in as one’s own original work constitutes cheating. Turning in a paper that was authored by another person in total or part also constitutes cheating.

GRADING

Success in this class is measured by the degree to which students can demonstrate effort, understanding, and critical analysis of the subject matter in assignments, exercises and in-class discussions. Regular attendance, some class participation, timely completion of acceptable assignments, and a good research proposal/exam will result in a B. Sloppy or late work, irregular attendance, and/or a mediocre proposal/exam will reduce the grade. Especially insightful in-class contributions, extra effort on assignments, and an outstanding research proposal/exam will increase the grade. The proposal or take-home exam will count as 100 points; written assignments are 75, 50, 75 points; proposal presentations 25 points, and attendance, reading reactions, and class participation combined 100 points. Final grades are determined on a regular % scale.

COURSE SCHEDULE OF TOPICS AND ASSIGNMENTS

Note: The readings and assignments are to be completed for the date under which they are listed

PART I: POLITICS AND SCIENCE

Week 1: August 30

Introductions, goals of course, entry questionnaire, discussion

► Start reading the assignments for next time and prepare reactions!

Week 2: September 6

The nature of science: “What’s politics got to do with it?”

Program evaluation, policy research, and applied social science research

· Meenaghan et al., Chapters 1 and 2

· Weiss, Chapters 1 and 2

· Edelman, Ira. 2000. "Evaluation and community-based initiatives." Social Policy (Winter): 13-23. (on BB)

Week 3: September 13

The various types of evaluation research

The role of theory in applied research: “Should we care about theory, assumptions, ideology?”

Turn in reaction to each chapter from Meenaghan et al. Chapters 1-5 (20 pts)

► In-class exercise 1: Evaluation design (Fix the city)

· Meenaghan et al., Chapters 3,4,5

· Weiss, Chapters 3, 4

· Adler, Marina A. 2002. “The Utility of Modeling in Evaluation Planning: The Case of the Coordination of Domestic Violence Services in Maryland.” Evaluation and Program Planning Vol. 25, pp.203-213. (Reserve)

► Read the Weiss glossary (p.327-339). Print out (and go over) a copy of Chapter 2 (pages 13-19 "Developing and using a logic model") of The Urban Institute’s Evaluation Guidebook, which is located on the Internet at

http://www.urban.org/UploadedPDF/guidebook.pdf

Week 4: September 20

The roles of the researcher or “how many hats can one wear simultaneously?”

Turn in reaction to Dunnagan and Andrews readings (10 pts)

► For SOCY and others doing a proposal: Proposal outline due

► In-class exercise 2: Create a logic or documents model

· Weiss, Chapter 5

· Rutman, Leonard and George Maowbray (eds.). 1983. Understanding Program Evaluation. Newbury Park, CA: Sage. Chapter 4. (Reserve)

· Meenaghan et al., Chapters 6 and 7

· Dunnagan, Tim et al. 2000. “Doing effective evaluations: A case study of family empowerment due to welfare reform.” Evaluation and Program Planning 23:125-136. (Reserve)

· Andrews, Arlene Bowers, 2004. “Start at the end: Empowerment evaluation product planning.” Evaluation and Program Planning 27:275-285. (Reserve)

PART II: ISSUES IN THE RESEARCH PROCESS

Week 5: September 27

Research design: strengths and weaknesses of design approaches

“Why use comparison groups?”

Turn in reaction to Grossman and Mock readings and Meenaghan et al. Chapters 8, 9 (20 pts)

► In-class exercise 3: Critique of the Havas MD WIC article (PDF on BB)

· Grossman, Jean and Joseph P. Tierney. 1993. "The Fallibility of Comparison Groups." Evaluation Review 17:556-571. (Reserve)

· Meenaghan et al., Chapters 8 and 9

· Weiss, Chapter 8

· Mock, Nancy B. et al. 1993. "The Utility of Case-Control Methods for Health Policy and Planning Analysis." Evaluation and Program Planning 16:199-205. (Reserve)

· Havas, Steven et al. 1998. “Final results of the Maryland WIC 5-a-day promotion program.” American Journal of Public Health 88: 1161-1167. (on BB) read this for in class exercise!

Week 6: October 4

Research design: the use of multiple methods (experimental and quasi-experimental designs)

“When to use experiments or quasi-experiments?”

► Assignment 1 peer review version (two copies) due: Evaluability assessment

► In-class peer review of Assignment 1

· Weiss, Chapter 9.

· Meenaghan et al., Chapter 10.

Week 7: October 11

Research design: measurement issues (Factor Analysis and Reliability procedures)

“Why be concerned about validity and reliability of indexes and scales?”

Turn in reaction to Özkan reading and Meenaghan et al. Chapter 10 (10 pts)

► Assignment 1 final version due

· Weiss, Chapters 6, 7, and 11

· Özkan, Türker and Timo Lajunen. 2005. “Masculinity, femininity, and the Bem Sex Role Inventory in Turket.” Sex Roles 52:103-110. (Reserve)

PART III: ISSUES IN STATISTICAL ANALYSIS

Week 8: October 18

Data analysis: Ordinary Least Squares (OLS) multiple regression analysis

Steps, interaction effects, squared terms

“Which statistics are appropriate for which data and which research question?”

· Allison, Preface, Chapters 1,2,4,5,6

► Those weak in statistics, please review:

Meier, Kenneth J. and Jeffrey L. Brudney. 2002. Applied Statistics for Public Administration. Chapters 1-6 and 9-14. (book on Reserve)

► In-class exercise 4: Table construction from computer output

Week 9: October 25

Data analysis: Comparing OLS and Dichotomous Logistic Regression analysis (DLR)

· Allison, Chapters 3,7,8,9

· Cizek, Gregory J. and Shawn M. Fitzgerald. 1999. “An Introduction to logistic regression.” Measurement & Evaluation in Counseling & Development 31: 223-245. (on BB)

· Weiss, Chapters 12 and 13

► In-class exercise 5: Rossi and Freeman Exercise 6-D

► In-class discussion of computer output

Week 10: November 1

Data analysis: OLS and DLR tables and interpretation of results continued

Turn in reaction to Harmon and Dukes et al. readings (10 pts)

COURSE CONTENT AND OBJECTIVES

General

Academic Misconduct

PART III: ISSUES IN STATISTICAL ANALYSIS

Thanksgiving: No Class! Enjoy!