>

CMQ Documents and Reports

> We realize that the issues involved in conducting high-quality job analyses — and especially, in linking job analysis data to bottom-line HR and personnel decisions — can at times be technically complex.

Many of our clients express interest in learning more regarding these technical issues, or obtaining more detailed background in the job analytic, test validation, measurement, or large-scale occupational analysis processes.

For those who want to learn more about developing defensible, valid organizational applications and collecting high-quality job analysis data, we have included links to the documents, reports, and research presentations shown below.

The index below organizes the papers into overall topic areas; you can download individual documents under each of these general categories.

Index to Document Categories

Development of the CMQ

These papers provide background on the overall design goals of the CMQ. The first describes the nationwide field test conducted to produce the first paper-and-pencil version, and the others provide a more recent summary of the way in which CMQ's scoring system has evolved, including its linkages to the Data-People-Things constructs of Functional Job Analysis. Back to topic list.

  • Harvey, R. J. (1993). Research monograph: The development of the CMQ. San Antonio, TX: The Psychological Corporation.
  • Harvey, R. J. (2004, April). Empirical foundations for the Things-Data-People taxonomy of work. In Fleishman, E. A. (Chair), Things, Data, and People: Fifty years of a seminal theory. Symposium presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Chicago.
  • Fine, S. A., Harvey, R. J, & Cronshaw, S. F. (2004, April). FJA strategies for addressing O*NET limitations in a post-DOT environment. In Fleishman, E. A. (Chair), Things, Data, and People: Fifty years of a seminal theory. Symposium presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Chicago.

Back to topic list.

CAT and IRT

The papers examine measurement and assessment issues involving Item Response Theory (IRT) and computer-adaptive testing (CAT) when assessing person-side characteristics of workers that may be linked to job analysis data. Back to topic list

  • Wagner, T. A., & Harvey, R. J. (2005, April). CAT Item Exposure Control for the Wagner Assessment Test (WAT). Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Los Angeles.
  • Harvey, R. J. (2005, April). IRT strategies for identifying rater quality in job analysis ratings. In VanIddekinge, C. H. (Chair), New evidence on individual differences in job analysis ratings. Symposium presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Los Angeles.
  • Trippe, D.M., & Harvey, R. J. (2003). Item Response Theory Analysis of the IPIP Big-Five Scales. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology.
  • Craig, S. B., & Harvey, R. J. (2004, April). Using CAT to reduce administration time in 360 performance assessment. In Craig, B. (Chair), 360, The next generation: Innovations in multisource performance assessment. Symposium presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Chicago.

Back to topic list.

Linking and Validation

These papers deal with issues involved in making valid and accurate linkages of job-side descriptions of work to person-side inferences of required worker traits. Back to topic list

  • Harvey (2006, May). Moving Beyond DOT and O*NET: How Do We Solve the Challenge of Linking Work Activities and Worker-Trait Requirements? Presentation made to the Personnel Testing Council of Metropolitan Washington, DC.
  • Wagner, T. A., & Harvey, R. J. (2005, April). JCV Predicting DOT Worker-Trait Requirements from CMQ Job Analysis Ratings. Paper presented at the National Conference of the Society for Industrial and Organizational Psychology, Los Angeles.

Back to topic list.

Faking and Data Quality

These papers consdider issues involved in detecting and dealing with faking in person-side assessments, as well as assessing the quality of job-side ratings. Back to topic list

  • Harvey, R. J. (2004, April). Quantifying job analysis rater quality using IRT appropriateness indices. In Ployhart, R, & Ford, D. (Chairs), Individual and subgroup differences on job analysis and competency ratings. Symposium presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Chicago.
  • Harvey, R. J. (2005, April). IRT strategies for identifying rater quality in job analysis ratings. In VanIddekinge, C. H. (Chair), New evidence on individual differences in job analysis ratings. Symposium presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Los Angeles.

Back to topic list.

Concerns Regarding O*NET and DOT

Numerous studies have documented very serious concerns regarding the psychometric quality and validity of both work-side descriptions and job-side inferences of personal trait requirements reported by DOT and O*NET. Back to topic list

  • Harvey, R. J., & Hollander, E. (2002, April). Assessing interrater agreement in the O*NET. In Wilson, M. A. (Chair), The O*NET:  Mend it, or end it? Symposium presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Toronto.
  • Harvey, R. J., & Hollander, E. (2004, April). Benchmarking rWG interrater agreement indices: Let’s drop the .70 rule-of- thumb. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Chicago.

Back to FAQ topic list.

Job-Component Validation (JCV) and "Synthetic Validity"

Issues involved in using "synthetic" methods for linking person- and job-side information, including job-component validation (JCV). Although we remain optimistic regarding the usefulness of JCV methods in some applications, a number of concerns still remain, especially in high-stakes assessment/selection situations. Back to topic list

  • Harvey, R. J. (2011, April). Deriving synthetic validity models: Is R = .80 large enough? Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Chicago.
  • Wagner, T. A., & Harvey, R. J. (2004, April). Job-component validation using CMQ and O*NET: Assessing the additivity assumption. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Chicago.

Back to topic list.

Learn more about what we do, review our Frequently Asked Questions (FAQ), or sign up and get started now!