APSY-UE 25                                                                                                                      Spring 2021

APSY-UE 25

RESEARCH METHODS I

PROBLEM SET #4

Instructions:

• This problem set aims to assess your understanding of the material covered in Lectures #1 to #20. It draws on the warm-ups, lectures, and recitations.

• You must turn in your answers by Wednesday, April 14, at 5pm EST electronically through the Assignments tab of the course site.

• You may complete the problem set with others, but you must write up your own answers.

• If you work with others in the class, you should acknowledge their contributions using a footnote stating “In collaboration with [First name] [Last name]”. This will help the teaching team understand similarities between assignments by different students.

• You may ask the teaching team questions about the instructions via e-mail or during office hours. You may not, however, ask for hints or whether your answers are on the right track.

• Once all problem sets have been graded, an answer sheet will be posted to the course site and you may request the teaching team to discuss the answers during office hours.

• Submit your answers to the questions as a .doc or .docx file with the name “[First name] [Last name] – Problem set 4”.

• Submit your Stata commands as a .do file with the same name. You do not need to turn in tables or graphs. You must turn in a single .do file, regardless of how many datasets you are using. Your .do files should call in the dataset(s) with which you are working (using the “use” command in Stata).

• If your answers involve computations, please show your as much work as you can.

• Remember to check your spelling and grammar before submitting your problem set. Problem sets with more than five spelling or grammar mistakes will be deducted five points.

• Respect the sentence limit for each question. Answers that go over the sentence limit will be deducted one point per question.

Read Duckworth, A. L. et al. (2007). “Grit: Perseverance and passion for long-term goals,” Journal of Personality and Social Psychology, 92(6), 1087-1101. You may access it at: https://bit.ly/3s2KPHk. Please, answer each question in no more than two sentences, referring to the page, column, and paragraph numbers that support each of your answers (e.g., “p. 1, col. 2, para. 2”).

1. On p. 1088, col. 1, para. 5, the authors describe “a wide range of achievement outcomes affected by IQ.” What type of validity evidence are they providing in favor of measures of intelligence? Explain. [10 points]

2. On p. 1089, col. 1, para. 5, the authors write “Grit overlaps with achievement aspects of conscientiousness but differs in its emphasis on long-term stamina rather than short-term intensity.” To what two types of validity evidence are they alluding? Explain. [10 points]

3. From p. 1090, col. 1, para. 4 to col. 2, para 1, the authors explain that each item in the initial version of the grit scale was scored on a scale that ranged from 1 (“not at all like me”) to 5 (“very much like me”). What is the scale of each item? Explain. [10 points]

4. In the second column of Table 1, the authors report the “item-total r.” (You are not expected to understand factor analysis to answer the two questions below).

a. Which statistic, reported in the Stata output for Cronbach’s alpha, does that column represent? [5 points]

b. Based on this statistic, which item would you drop from the initial version of the grit scale? Explain. [5 points]

5. In the first column of Table 2, the authors report the Cronbach alpha statistics for six administrations of the initial version of the grit scale. What type of reliability do these numbers measure (i.e., inter-item, inter-rater, or both)? And what is the scale for this statistic (i.e., what are the minimum and maximum values it may take)? Explain. [10 points]

6. In the fourth column of Table 3, the authors report the “% of variance in success explained by grit.” What statistic that we learned in class are the authors reporting? Interpret the value for the first row of this column (i.e., the value of that statistic for educational attainment among adults aged 25 and older for study 1). [10 points]

For the questions below, you will use the ratings that 20 teachers were assigned on a four-item classroom observation protocol devised to measure their rating effectiveness. You may access it at: https://bit.ly/2PNwT6S. Each item is scored from 1 (“needs improvement”) to 5 (“exemplary performance”).

7. Using Stata, generate a variable called “odd_sum” in which you sum the scores for the oddnumbered items and a variable called “even_sum” in which you sum the scores for the evennumbered items. Use both scores to compute the split-half reliability for this protocol. (Do not forget to perform the Spearman-Brown adjustment). [10 points]

8. Using Stata, obtain Cronbach’s alpha for this protocol. Would you prefer to use the approach in the previous question, or the approach used in this question, as a measure of the inter-item reliability of this classroom observation protocol? Explain why. [10 points]

9. Using the Stata output for question 8, are there any items which you can drop to improve the reliability of this classroom observation protocol? If so, identify that item and explain why you selected it. [10 points]

10. Using the Stata output for question 8, explain why the item-test correlations are consistently higher than the item-rest correlations. [10 points]