Resources

Resources for Measurement

Articles

• Loken, E., & Gelman, A. (2017). Measurement error and the replication crisis. Science, 355(6325), 584–585. https://doi.org/10.1126/science.aal3618 https://www.science.org/doi/pdf/10.1126/science.aal3618

• Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393 https://journals.sagepub.com/doi/pdf/10.1177/2515245920952393

• Intraclass Corrrelations

  • McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46. https://doi.org/10.1037/1082-989X.1.1.30
    • https://www.researchgate.net/profile/Kenneth-Mcgraw/publication/232568057_Forming_Inferences_About_Some_Intraclass_Correlation_Coefficients/links/00463538728bc80398000000/Forming-Inferences-About-Some-Intraclass-Correlation-Coefficients.pdf
  • Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420
    • https://www.researchgate.net/profile/Marcelo-Alves-8/post/Is-a-repeated-measures-ANOVA-the-right-statistical-test-to-use-for-the-analysis-of-my-data/attachment/59d630b379197b807798e8c7/AS%3A362744118300682%401463496346017/download/ICC.pdf
  • Ten Hove, D., Jorgensen, T. D., & Van Der Ark, L. A. (2024). Updated guidelines on selecting an intraclass correlation coefficient for interrater reliability, with applications to incomplete observational designs. Psychological Methods, 29(5), 967–979. https://doi.org/10.1037/met0000516

• Multilevel/longitudinal reliability

  • Lai, M. H. C. (2021). Composite reliability of multilevel data: It’s about observed scores and construct meanings. Psychological Methods, 26(1), 90–102. https://doi.org/10.1037/met0000287
    • https://quantscience.rbind.io/postprint/Lai_2020_pm_mcfa_reliability_am.pdf

Data

• Open-Source Psychometrics Project
  • Data for public use

Reporting

• APA Style JARS (see sections related to measures, instrumentation, and psychometrics): https://apastyle.apa.org/jars/quant-table-1.pdf
• Carpenter, S. (2018). Ten steps in scale development and reporting: A guide for researchers. Communication Methods and Measures, 12(1), 25–44. https://doi.org/10.1080/19312458.2017.1396583
• Jackson, D. L., Gillaspy, J. A., & Purc-Stephenson, R. (2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14(1), 6–23. https://doi.org/10.1037/a0014694
  • https://web.archive.org/web/20150518131940id_/http://www3.nd.edu/~kyuan/courses/sem/readpapers/Jackson-reportingSEM09.pdf
• Brown, T. A. (2015). Confirmatory factor analysis for applied research. The Guilford Press.
  • Chapter 4 (Example reporting of CFA in Appendix 4.3)
• FDA guidance on item response theory
  • https://www.fda.gov/regulatory-information/search-fda-guidance-documents/submitting-clinical-trial-datasets-and-documentation-clinical-outcome-assessments-using-item