In this paper we compare and contrast student’s pretest/post-test performance on the Halloun-Hestenes force concept inventory (FCI) to the Thornton-Sokoloff force and motion conceptual evaluation (FMCE). Both tests are multiple-choice assessment instruments whose results are used to characterize how well a first term, introductory physics course promotes conceptual understanding. However, the two exams have slightly different content domains, as well as different representational formats; hence, one exam or the other might better fit the interests of a given instructor or researcher. To begin the comparison, we outline how to determine a single-number score for the FMCE and present ranges of normalized gains on this exam. We then compare scores on the FCI and the FMCE for approximately 2000 students enrolled in the Studio Physics course at Rensselaer Polytechnic Institute over a period of eight years (1998–2006) that encompassed significant evolution of the course and many different instructors. We found that the mean score on the FCI is significantly higher than the mean score on the FMCE, however there is a very strong relationship between scores on the two exams. The slope of a best fit line drawn through FCI versus FMCE data is approximately 0.54, and the correlation coefficient is approximately r=0.78 , for preinstructional and postinstructional testings combined. In spite of this strong relationship, the assessments measure different normalized gains under identical circumstances. Additionally, students who scored well on one exam did not necessarily score well on the other. We use this discrepancy to uncover some subtle, but important, differences between the exams. We also present ranges of normalized gains for the FMCE in a variety of instructional settings.