Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

PSYC151 (UCSD)--Testing and Measurement

Paper #2: Psychometric Analysis/Interpretation

For this paper SPSS output based on psychometric testing of a work/organization-related dataset will be provided.  The dataset (in SPSS format) will be provided in case you have access to SPSS, but for your reference, following are the items and scales:

Note that I computed an average scaled score for each domain (e.g., jobsat_sc = mean(js1, js2, js3).  Following is an example of the 6-point Likert scale used for the self-report items:

 

Following is statistical output preceded by the psychometric instructions.

(1) Reliability

 For each of the domains assess the internal consistency via Cronbach's Alpha (α). Give a general description/summary...how did we do? Is the reliability of the score acceptable?  Any hints as to items that are problematic (i.e., possible candidates for deletion?.....or are there any that are omitted that may have improved the estimate?)….pay attention to corrected item-total correlations as well as item intercorrelations.

So, overall what is your opinion of the reliability of the scales?  Based on what you've learned from your reliability analysis would you recommend this instrument for future use?

Job Satisfaction

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Social Support at Work

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Organization Commitment

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Locus of Control at Work

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Job Environment

 

 

 

 

 

 

 

 

 

 

 

 

 (2) Predictive Validity

 

 In the spirit of predictive validity, you are going to interpret a simple linear regression (one predictor) and then run the same model adding a 2^nd predictor, predicting "overall, I would recommend my company for employment" (see table for outcome below).  Please detail the results of significance testing (α = .05) and summarize the correlations as well as variance explained (i.e., R²) and put that in the context of model fit for each of the below two regression models..   Also, address if the impact of the domain score that was used in both models had a change once it was incorporated in the multiple regression (was it significant in one model but not another?).  Did the model fit improve once the 2^nd predictor was added (i.e., is there a substantive difference in the R² when the 2^nd predictor is added)? Overall, what is your conclusion as to the predictive validity of the predictor/predictors?  What do you see as the practical implications of your regression model(s)?

 

Criterion

 

 

 

 

 

Predictive Validity: Simple Linear Regression

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Predictive Validity: Multiple Linear Regression

 

 

 

 

 

 

 

 

 

 

 

 

(3) Construct Validity

 

(a) Construct Validity (Correlations)—Following are the correlations for the domain/scale scores.  Is there preliminary evidence that they are measuring separate constructs (i.e., discriminant validity) or is there evidence otherwise (i.e., is there substantial overlap such that they may be measuring similar constructs)? This may help shed some insight to the results for the following exploratory factor analysis.

 

 

(b) Construct Validity (Exploratory Factor Analysis)--next we are testing the factorial structure of the instrument (which is at the item level).  For this section, you will only interpret the rotated matrix.  There is no firm guideline as to what  constitutes a sufficiently high factor loading, so for now just locate the component that has the highest loading for the given item and use that for interpretation (keeping in mind that there may be some items that have nothing but low component/item correlations or some that will cross-load).  What is your interpretation?  Did the solution mirror the proposed structure of the domains?  If not, please speculate why that was the case.    

 

 

(c) Construct Validity (between group comparisons).  We are now going to test mean differences on the “overall” item comparing one of the between-group demographics.  We have run a one-way ANOVA, and if you had an a priori hypothesis what pattern of means would you have anticipated (i.e., which group has the higher/lower mean on the DV?).  Detail if (1) the group means are in the proposed direction and (2) do we have significant differences? (level of significance = .05).  Also, point to model strength via the effect size (i.e., partial eta squared with small/medium/large being, for purposes of this assignment, .01/.059/.138), Draw your conclusion.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Additional Note:

 1) For discriminant validity we will use r < .35 as preliminary evidence of discriminant validity.

2) For the EFA we will use component/factor loadings > .45 as possible items used for component/factor interpretation.

For both of those guidelines those are only general cutoffs.....as we discussed in class, e.g., if you have a component with quite a few loadings > .65 and one item that has a factor loading of .46 BUT that same item has a loading of .72 on another component then it is clear you should use the .72 and the accompanying component for interpretation.

PS. And if it illuminates your data/results/interpretation you are welcome to use a graphic (error bar, bar chart etc.) for the one-way ANOVA (construct validity)