SOCIOLOGY 110: Research Methods                                                     Spring, 2022
 
GUIDELINES for RESEARCH PROJECT PROPOSAL PAPER
 
This course in Research Methods has two objectives: 1) for students to gain an understanding of contemporary social science research, and 2) for students to practice strategies and skills for gathering/analyzing information, and developing/writing a coherent grounded research proposal.
 
Students will get direction and feedback at each step of proposal preparation (i.e., selecting a topic, obtaining/analyzing/synthesizing literature, proposing a project).  You are strongly encouraged to meet with the TA/Instructor to get individual feedback.  The final paper grade reflects the quality of the final product, but also the development shown over the quarter.
 
Project Proposal Paper (25%)
The final paper will be based on an analytical literature review of a research topic in sociology, and a proposed project to further social science inquiry on that topic.  The course paper will consist of an introduction, background/literature review, proposed methods and data analyses, discussion and conclusions and will be outlined in the last section.
 
Exercise 1 and consultation meeting (4%)
Select a topic of inquiry.  Obtain 6 social science scholarly research articles on the topic, include three based on quantitative (survey) research designs, and three based on qualitative field (participant observation, ethnography, focus groups, content analysis) research designs.  Experimental and Archival research are not included in this course exercise.  Provide an annotated bibiliography of these articles.
 
Exercise 2 (8%)
          Detailed methodological analysis of each survey research article, and an integrative summary of the three articles.  Create proposed alternative method including a short questionnaire (6 items).  You will pre-test the questions on a few respondents and report your findings about the instrument in your final proposal paper.
 
Exercise 3 (8%)
          Detailed methodological analysis of each field research article, and an integrative summary of the three articles.  Create proposed alternative method including an interview guide (with one brief pilot intensive interview) and a field guide (with one brief pilot systematic observation).  You will pilot test these guides and report your findings about the instruments in your final proposal paper.
 
Further details on these Exercises are provided in the following pages.
 
Exercise 1 (DUE April 14)
Select a topic of inquiry.  Obtain 6 social science scholarly research articles on the topic, include three based on quantitative (survey) research, and three based on qualitative field (participant observation, ethnography, focus groups, content analysis) research.  Provide a brief annotated bibliography of each of these articles.  Additional articles are permitted for the final proposal paper (eg., experimental or archival research) when appropriate.
 

1. Your introductory paragraph describes your research topic, the main concept(s) you are interested in studying, and the scope of previous research.

 
Describe why it matters.  Opening statements might address some of the following issues: Why is it worth studying? Why is this an interesting topic to a sociologist (not “to society”)?  Is the study of primarily practical and/or theoretical significance? What is generally known or of concern about the topic?  What, specifically, do you hope to contribute with your research?  You must provide indication that the topic is sociological (how it engages sociological perspectives or theories).
 
Then, state the problem/research question.
This should generally take one of the following forms: “What are the causes of y?”, “What are the consequences of x?”, “Is there an association between x and y?”  “How does x influence y?” “To what extent does x impact y?”
 
For the quantitative portion of your mixed methods design, you will need to clearly specify your hypothesis linking x and y.  Include a definition of your measurable independent and dependent variables (these must be measurable) and the ideas linking them (why/how they are related must be sociological).  For this course, your research hypothesis (and your quantitative articles) must be testable with a survey method.
 
For the qualitative portion of your mixed methods design, you will need to clearly describe your research focus.  What specifically do you want to learn more about in depth – who, what, when, where, and perhaps why?  For example, what are the causes, consequences, or process of x or of y?
 
Thinking Outline

• Statement of the “problem.” I overheard a conversation about…
• Why important. I found the topic (theoretically, sociologically) interesting for the following reasons…
• Theoretical Perspectives. Several people chimed in on various aspects of the topic.
• One individual talked about it in this way….
• Another person looked at it from a different angle…
• Someone else added the following insights…..
• Entry point. Finally I spoke up to add my “two cents” and explained what appeared to be lacking/needed correcting in what was said….

 

1. The annotated bibliography will then include a very short descriptive paragraph (your notes about the article in your own words) on each of the SIX sources (three quantitative and then three qualitative) that you have so far obtained related to your topic. These sources should be published within the last ten (preferably five) years.

 
Clearly label which of your sources are quantitative and which are qualitative.
 
An annotated bibliography is a list of citations (articles, books, documents).  Each citation is followed by a concise descriptive and evaluative paragraph (approximately 150 words) called the annotation.  The purpose of the annotated bibliography is to inform the reader of the relevance, accuracy, and quality of sources on your list.  It is also used to jog your memory about articles you have already read as you continue forward with your literature review and research design preparation.
 
It should include:
A full citation that includes the author, date, title, and publication information.
Following, a brief paragraph addresses

• The research question, theoretical framework, and general methodological design of the study,
• Significant findings and substantive conclusions of the study (ONLY as regards YOUR research question),
• A brief, critical analysis of the quality (strengths and weaknesses) of the study.

 
Proper citation: 
The American Sociological Association publishes a “Quick Style Guide,” which provides all the information you need to cite properly both in-text and in your bibliography.  The Guide can be found on the course website (and cited in your text).
 

• See next page for an example –

 
 
An example:
Waite, L.J., Goldscheider, F.K., and Witsberger, C. 1986. “Nonfamily Living and the Erosion of  Traditional Family Orientations among Young Adults”. American Sociological Review. 51:541-554.
 
“The authors, researchers at the Rand Corporation and Brown University, use data from the National Longitudinal Surveys of Young Women and Young Men to test their hypothesis that nonfamily living by young adults alters their attitudes, values, plans, and expectations, moving them away from their belief in traditional sex roles.  They find their hypothesis strongly supported in young females, while the effects were fewer in studies of young males.  Increasing the time away from parents before marrying increased individualism, self sufficiency, and changes in attitudes about families.  In contrast, an earlier study by Williams cited below shows no significant gender differences in sex role attitudes as a result of nonfamily living.”
 
Notice that in this example, particular attention is given to gender differences, presumably because the reviewer’s research question focuses on gender differences among young adults.  In addition, the last sentence notes an important contradictory finding in a similar study.  Think of the annotations as representing a “conversation” between the authors of the various articles you are using.  Comparing and contrasting the three studies will assist you in identifying the important themes/issues/problems associated with your research topic.  While not reflected in this example, the discussion between the studies might also involve competing theories or paradigms.
 
 
Exercises 2/3
Now that you have developed your research question, you need to begin the process of developing your research design.  Since your proposal will ultimately involve a multiple methods proposal, recognize that the process will differ for your qualitative and quantitative studies.  You will move toward the development of your proposed design by first providing a more detailed analysis of the research articles on your chosen topic (from Exercise 1).
 
Exercise #2 (DUE May 12) Quantitative Survey Research

1. You will complete Parts I-IV for each of your quantitative/survey research articles. The issues you need to address are outlined below, and your critical analysis will be approximately three typed pages for each article.

 

1. Theoretical Orientation.

What is the research question/topic?  What theory is presented?  Why is the research important?  What are the hypotheses?
 

1. Methods.
2. What is the unit of analysis? What is the time dimension? What sampling procedure was used? Was it appropriate for the research question/ the population under consideration?  Discuss the sample size (and demographic characteristics), and representativeness.

 

2. What are the independent and dependent variables? What other key (control) variables are identified? How are the central concepts defined and operationalized? (Include the names/definitions, and indicators of each major concept.)  Is the level of measurement appropriate? How does the study assess reliability and validity?

 
III. Data Collection and Analysis Critique

1. What was the response rate, and does the author address the issue of nonresponse bias? Do the questions asked of respondents (number, wording) adequately reflect the concepts? For closed-ended questions, were the response categories appropriate, exhaustive, and mutually exclusive?  For open-ended questions, how were the responses categorized?  Were the questions clear and unambiguous, avoiding social desirability, etc.?

 

2. What statistical analyses were are applied to the data? What are the key research results? Are the findings clearly presented, with appropriate inference about the strength and direction of the association and statistical significance of hypothesized relationships? Does the author make causal inferences?  What evidence is presented regarding direction of influence and nonspuriousness?

 
 

1. Conclusions and Reporting.

Is the method used appropriate, or well-suited, to addressing the research question or testing the hypothesis?
How do the researchers address ethical concerns?  Are procedures implemented that adequately protect participants’ rights and welfare?
What substantive conclusions are drawn from the study?  Are the conclusions appropriate to the design and the representativeness of the sample?   Does the author discuss alternative explanations of findings or results? Does the research replicate, modify, extend, or contradict previous studies?
Has the researcher reported any shortcomings in the study design or execution?  Does the author discuss the implications of study results for future research?
What do you think of the quality of the research?
 
RECOMMENDED: Construct a matrix with rows summarizing the Theory, Methods (population, sample, concepts, variables), Findings, Conclusions, Limitations, with the three articles on the columns.  This will greatly facilitate your ability to compare across the three articles for the next section:
 

1. Proposed Research Design – this is the MAIN POINT of the exercise.

This section will be approximately four typed pages.
 
First, provide a critical integration (compare and contrast) of the articles to summarize “what is known” about the topic.  You might note important similarities or differences in theoretical orientation, sampling/methods, data analysis/results, conclusions, strengths and limitations.  Taking the articles together, what do you now know overall about your research topic, and what gaps in existing knowledge remain?
 
Then, propose an original survey design to replicate, extend, or improve upon the studies you reviewed, particularly addressing any noted errors, limitations, or deficiencies.
What population/sampling procedure would you use; why? What mode of survey administration will you use: why?  How will you define your main concepts? Will you use the same or different questions as those in your literature; why?
Include your questionnaire.  You will need at least two indicators (or an existing scale) each for your independent variable, dependent variable, and control variables.
 
Important: AFTER these are approved, you will “pre-test” them on about five cases, and you will include the results (a summary of what you learned) in your final proposal paper as an Appendix.  You will not be collecting data.  The pre-test is for the purpose of checking that the questionnaire questions are understandable and can elicit the kinds of information needed to address the research questions, and should include at least five people (in person, email, discord, whatever).  You will not be statistically analyzing the data, but rather considering whether the questions are valid – this will be discussed in your paper.
Exercise #3 (DUE May 24) Qualitative Field Research

1. You will complete Parts I-IV for each of your three qualitative research articles. The issues you need to address are outlined below, and your critical analysis will be approximately two typed pages for each article.

 

1. Theoretical Orientation.

What is the research question/topic?  What theory is presented?  Why is the research important?  What are the hypotheses (if any)?
 

1. Methods.

What is the unit of analysis?  What sampling procedure was used (if any)?  Was it appropriate for the research question/ the population under consideration?  What was the research setting? How was it appropriate for the research topic?
 
III. Data Collection and Analysis Critique

1. Has the research set out to test hypotheses or generate theory? How are the main concepts/variables ultimately defined and measured? How does the study assess reliability and validity? How were people interviewed/observed?  Evaluate the interview/field guides.  How does the author discuss relationships with participants and reflexivity?  How much did the researcher participate-observe, or reveal their identity (overt-covert)?  How might the researcher’s feelings, experiences, or background have influenced the observations and interpretations?

 

2. What form of qualitative analysis does the author apply? Are the analytic techniques (e.g., coding) adequately described? Does the analysis lead to an in-depth understanding of the topic?  Does the author discuss member checking and/or negative cases?

 

1. Conclusions and Reporting.

Is the method used appropriate, or well-suited, to addressing the research question or testing the hypothesis?
How do the researchers address ethical concerns?  Are procedures implemented that adequately protect participants’ rights and welfare?
What substantive conclusions are drawn from the study?  Are the conclusions appropriate to the design?  Does the author discuss alternative explanations of findings or results?  Does the research replicate, modify, extend, or contradict previous studies?
Has the researcher reported any shortcomings in the study design or execution?  Does the author discuss the implications of study results for future research?
What do you think of the quality of the research?
 
RECOMMENDED: Construct a matrix with rows summarizing the Theory, Methods (sample, setting, guides, concepts), Analysis/Findings, Conclusions, Limitations, with the three articles on the columns.
 

1. Proposed Research Design – this is the main point of the exercise

This section will be approximately three-four typed pages.
 
First, provide a critical integration (compare and contrast) of the articles to summarize “what is known” about the topic.  You might note important similarities or differences in theoretical orientation, sampling/methods, data analysis/results, conclusions, strengths/limitations. Taking the articles together, what do you now know overall about your research topic, and what gaps in existing knowledge still remain?
 
Then, based on these gaps, propose an original qualitative design to replicate, extend, or improve upon the studies you reviewed, particularly addressing any noted errors, limitations, or deficiencies.  What population/sampling procedure would you use; why? What qualitative design will you use; why?  How will your qualitative design (intensive interview plus participant-observation) differ from the studies you reviewed?  What type of individual will you interview; why? What research setting/observations will you use; why?
Include your tentative interview guide (an introductory statement, prepared general questions/probes) and your tentative field guide (who/what you will observe in what setting).
 
IMPORTANT: AFTER your tentative interview guide and field guide are approved, you will “pilot test” one brief interview and one observational session.
 
The pilot test for your qualitative design includes a ten minute intensive interview (with the two or three broad questions/prompts; include thoughts, feelings, experiences, behaviors); the interviewee does not have to be a member of your specified population, just anyone you can chat with and record.  You are required to provide the transcription (in an appendix), and to spend some time practicing analyzing it (last pages of the guidelines provide suggestions).
The pilot also includes a ten minute observational session (again, you were to outline the guide for the exercise), and you should submit your field notes also as an appendix.   Again, it does not have to be the kind of setting you would use in your  proposed study if this is not possible – the purpose is to see if the interview/field guides work to provide the kind of information you need to address your research topic.  If circumstances do not permit you to be among people, you might even have to try watching ten minutes of a video.
 
In your proposal paper (Not this exercise), you will include a summary of what you learned in your pilot study, and in an appendix a transcription of your brief intensive interview, and your field notes from your observation.  Pilot studies are standard for a scientist; you are conducting a validity check for your instruments and analysis strategies to inform/improve your project proposal (not conducting the proposed project).
 
 

FINAL Proposal Project PAPER GUIDELINES

 
Final Paper –You will submit an integrated course paper based on your critical analyses and synthesis of empirical studies, and proposed project design to further investigate your research topic.  Do not write your paper as though you have already conducted the project; you have conducted only a pre/pilot-test.
 
Your paper must include the following: I. Introduction II. Literature Review III. Method and Data Analysis proposal (both quantitative and qualitative) V. Discussion and Conclusions.  Detailed guidelines are provided following.
 
General REQUIREMENTS
 

1. The final course paper is to be approximately sixteen pages, double-spaced, typewritten paper. See Writing Guidelines for specifics. It will be structured as outlined below.
2. References. A minimum of six references of empirical studies on your topic as specified in Exercise 1.
3. Format. The paper is to be written in ASA style in three ways:
4. Bibliographic references at end of paper
5. Use of references in text
6. Use of headings – major headings are centered and in capital letters, sub-headings are in lower case letters, underlined, and to the left side.

 
 
STRUCTURE (USE these headings and sub-headings)
 
INTRODUCTION
(2 pages – 10 points)
 

1. What are you studying? Clearly specify your research question and its sociological importance in the first paragraph of the paper.  (This is a more developed version of your introductory paragraph from Exercise 1.)

 

2. What is generally known about this topic? How is your research question related to prior research and theory?

This portion is a very brief version of the issues you will address in the Literature Review.  In order to place your study in context, identify/develop at least two themes, issues or problems that relate to the broader topic you are investigating; your study will contribute to the literature in addressing these issues.  You can provide demographics, descriptive information, an historical overview of the topic, or other background material in this section if you choose.
 

3. HINT: Include that you will be using a “multiple-method” approach in designing your research, and discuss why/how this could be beneficial.

BACKGROUND/LITERATURE REVIEW
(5 pages – 34 points)
 
Review of your six (minimum) empirical studies.  The information will be drawn from your previous Exercises/feedback, but you will describe only the relevant points from each article to include in your literature review.   Again, do NOT just summarize the articles or include the pages from your exercises; write only about what matters (themes, issues, or problems) for your proposed research.
 
For each study, cover the following in about one-two paragraphs (4 points each):

1. What research questions or hypotheses did the researchers study? (Identify article by author and (date) of study.) As appropriate to your themes, BRIEFLY describe: the research methods used, characteristics of the sample, main findings and conclusions (as they relate to their main research questions or hypotheses), and your critique (strengths and limitations of the study).  This is a much refined version of your initial annotation.  Focus on the themes/issues you have stated in your introduction.  Notice this is a significantly shortened presentation revised from Exercises 2/3 which will focus only on the information relevant to the design of your proposed research.

 
Then, in about two-three pages (10 points):
SYNTHESIS – putting it all together.

2. Compare and contrast the theories/findings/conclusions in the various studies, and how they went about contributing explanations about your topic.  In what important ways are the studies similar/different?  What are the unique quantitative and qualitative contributions?  Taking all six of the articles together, what do you now know overall about your research topic, and what gaps in existing knowledge still remain?  This is an improved version from part B of your exercises.

 

3. In the last paragraph of the literature review, specify the research questions/hypotheses to be investigated in your proposed research based on the findings of your literature review and your thoughts on your topic.

 
 
METHODS AND DATA ANALYSES
(10 pages – 40 points)
 
Since this is a research proposal, you will specify how you would do the study; a multiple method approach is required.  You must describe how you would use each method to draw your sample/select the setting, collect your data (how will you get it, and what will it look like?), and analyze your data (what is the data analysis process, what will the results look like, and how might you present them).  (This is a much more developed version of your proposed design from Exercises 2/3.)
 
 
Quantitative Research Design
Your proposed survey research design will be approximately four pages. You must describe how you would use a quantitative survey method (sampling, data collection, analysis, etc.), focusing on the issues covered in the course material and Exercise 2.  Be sure to address concerns unique to quantitative methods, including ethical issues.  Be as specific and detailed as possible within the suggested pages. (10 points)
Include a copy of your final questionnaire (in the Appendix).  Summarize what you learned about your questions and responses from your pre-test data, noting any necessary revisions in your instrument or survey design. Describe appropriate data analyses procedures.  (10 points)
 
 
Qualitative Research Design
Your proposed field research design will be approximately six pages.  You must describe how you would use qualitative methods, focusing on the issues covered in the course material and Exercise 3.  Be sure to address concerns unique to qualitative methods, including ethical issues.  Be as specific and detailed as possible within the suggested pages. (20 points for design, analysis/coding)
 

1. Your fieldnotes from your pilot test will include the following (10 points):

 
Describe When/Where you conducted the interviews/observations?  How did you gain access, if necessary?  How did you locate informants or establish roles and relationships (develop rapport) with your participants?  What questions did you ask, in what order?  How long did the data collection take?  How did you record the data?  Note any events during the data collection, and characteristics of the setting.
 
For intensive interviews, you will include a complete transcription (typed word for word record of your recorded interview).  For observations, you will need to submit your complete raw field notes (including transcriptions of conversations/casual interviews).  These should be in an Appendix.
 
Was your social location similar to/different from your participants?  How did that shape the experience and the data you collected?  How did you feel as the researcher?  How well could you listen/observe?  How did your participants react?  Did you encounter any challenges?  What ethical considerations did you take into account?
 
Describe your data analyses procedures.  Read your literature review and all field notes for guidance.  Consider the following: For the initial pattern-seeking/coding, go line-by-line.  Ask of each discrete event, behavior, or place: 1. What does this represent?  What it is this an example of?  What is going on (what are people doing, or saying)? What are the assumptions made?  Pay attention to meaning, tone, frequency, and emphasis.
 
In your focused coding, make a list of all of the categories/themes that emerge (including subcategories).  Think about the relationship between the different categories.  Compare the data from your observations/informants.  As you stay focused on the research question, make notes of the most important similarities and differences between them.
 

2. Now, clearly describe your finalized qualitative design – sample, setting, interview/field guides, proposed analysis strategy, etc. (10 points).

 
 
DISCUSSION AND CONCLUSIONS
(3 pages –14 points)
 

1. Briefly summarize the expected contributions of your proposed research, including results, interpretation of the results, and the limitations of each method.

 

2. Synthesize the results, comparing and contrasting each method in terms of the overall product (how will the findings fit together or complement each other?).

 

3. Conclude with a statement of what you hope to discover about your subject matter. Summarize what you learned about your topic in terms of the current state of theory and research findings.  What issues remain to be investigated?  How will future theory (and application of alternative theoretical paradigms) need to be refined?

 
 
REFERENCES
(as many pages as you need to cite all sources – 2 points)
Follow ASA bibliography format.
 
 
 
WRITING GUIDELINES
All papers must be typewritten and double-spaced with margins of 1” all around. All papers should have a cover page stating the name of your topic, your name, the class name or number and the date.  Papers must be written in a size 12 font, using Ariel or Times New Roman print.  The cover page should also be in size 12 font – not in boldface print.  Pages must be numbered in the upper right corner.  The cover page is page one (1).  Do not show the page number on this page. The first typed page is page two.
 
Do NOT use any direct quotes from articles, text etc. in your paper without explicit quotation and citation, and even these should be used sparingly.  Paraphrase the author’s key ideas in your own words (and cite the author, year).  If you copy word for word without citation (parentheses, author, date and page number), you are committing plagiarism.  This is against the law.  Reference format must follow the ASA format and style of references in text of your paper and on your list of references at end of paper.
 
Clarity of Communication: A well-written paper communicates ideas of information in a clear-cut, understandable-to-the-reader style.  Poorly written papers, even when written in English, slow the reader down so that he/she may have to “translate” the paper.  This distracts the reader from the basic ideas of your paper.  Follow the steps necessary for you personally to avoid spelling, grammar, punctuation, sentence construction, and syntax (wording, word order) errors.
 
Papers not meeting these specifications will have their grade lowered.
 
Coding Qualitative Interviews – Suggestions for the Process
 
Steps to Successful Coding:

• Read the textbook!
• Get reacquainted
  1. Re-read your literature review. Make sure that you know what you are trying to study!  It is more important at this point that you know your dependent variable.  You are not wedded to an independent variable at this point.

 

• Shake off your biases

Do not “fish” for your desire outcome, But, don’t let a fear of bias immobilize you!
 

• Take a walk in your data

Listen to the recording again.  Read your transcription at least 3 times.  When you can anticipate what they will say next, you are ready to start coding.
 

• Initial Coding
  1. Go line-by-line. Ask of each discrete event, behavior, or place:
     1. What is this? What does it represent?
     2. What is this an example of?

• What is going on? What are people doing?  What is the person saying?  What are the assumptions made?

1. Pay attention to meaning, tone, frequency, and passion
2. Make a list of all of the categories/themes that emerge. There can be subcategories also.
3. Think about the relationship between the different categories and jot memos to remember
4. Keep the list on a separate sheet of paper. Assign a color or symbol to each category that you will use in your coding.

 

• Focused Coding
  1. Build on initial coding. Select a number of the core initial codes.
     1. You will discard many of the original codes and combine others. This is normal.

 

• Memo to self
  1. Note your reflections on deeper meanings, relationships between categories/themes.
  2. Note ideas about the potential social types you are creating.
  3. Note quotes that poignantly highlight certain themes or categories.
  4. Note exceptions, or things that appear to conflict with the theory you are creating.

 

• Create your types
  1. Compare Informant/Observation #1 to Informant/Observation #2 transcriptions and memos.
  2. As you stay focused on the research question, make note of the most important similarities and differences that arose between the informants.
  3. Create social types for your informants.
     1. Clearly define the types using the characteristics and themes you coded.
     2. Create a pictorial image of your types.

• Use quotes to support your argument.