PSYC 520 Midterm Study Guide

Total time: 80 minutes

~23 Multiple-Choice Questions

Cover matierals up to Reliability, chapter 8 of MT

~5 Short-Answer/Essay Questions

Questions similar to the following:

Define the term Test as discussed in the class.
What is the definition of measurement according to (a) Steven’s definition and (b) Michell’s definition? Give an example that satisfies (a) but not (b).
What is the difference between standardized and normalized scores?
Calculate the percentile ranks given the score frequency table (similar to Table 2.3 of the text).
A researcher is interested in creating a test for [some psychological construct]. Based on your knowledge of the test development process, provide four guiding questions that the researchers should consider, and explain why each question is important.
Based on the guidelines in your text, suggest two ways to improve the following multiple-choice item, and explain your suggestions. [you may see a different item in the exam.]
Which of the following is not a reason for conducting factor analysis?
1. Factor analysis may be unstable in small samples
2. Factor analysis can provide insights on the underlying structure of the items
3. Factor loadings are used for selecting items
4. Factor analysis can provide evidence for the initial test blueprint
Using a construct of your choice, describe the procedure of developing a Thurstone scale.
Name two types of response distortion, and explain each of them.
The formulas for difficulty (P) and discrimination (D) of an item are:

\(P\) = Number of correct responses / Total number of responses
\(D\) = \(P_\text{upper}\) - \(P_\text{lower}\)
1. Calculate the difficulty and discrimination values for the following hypothetical multiple-choice items. An asterisk next to a response option indicates it is the correct answer.
  
  Item 1:
```
       A  B  C  D*
Lower  9  4 11 14
Upper  0  0  0 35
```
  Item 2:
```
        A  B  C  D*
lower  11 10  7  8
upper  23  0  2 10
```
2. Which item is performing better?
What is random error? What is systematic error? Explain how each type of error is related to reliability in measurement theory.
What is true score in classical test theory (CTT)?
What is the definition of reliability in CTT? If you write the formula for reliability calculation, explain what that means.
What are the assumptions in calculating test-retest reliability and alternative forms reliability estimates?
What are the assumptions in calculating Spearman Brown prophecy formula for reliability estimate? When do you use this estimate?
The following shows the covariance matrix and the means for four items [below is just an example]:

1 2 3 4 Mean

Item 1 3 3

Item 2 1 4 3

Item 3 1 1 2 3

Item 4 1 1 1 3 3

Are the items parallel, tau-equivalent, essentially tau-equivalent, or congeneric? (Select the one that is the most restrictive.) Explain your answer.
Derive the alpha coefficient for reliability, under the assumption of a set of essentially tau-equivalent items.
Given the following R output, write a sentence to report the reliability as if you’re doing it for a manuscript.

	1	2	3	4	Mean
Item 1	3				3
Item 2	1	4			3
Item 3	1	1	2		3
Item 4	1	1	1	3	3