Evaluating professionalism in medical undergraduates using selected response questions: findings from an item response modelling study

dc.creator	McLachlan John C
dc.creator	Finn Gabrielle M
dc.creator	Tiffin Paul A
dc.date	2011-06-01T00:00:00Z
dc.date.accessioned	2015-07-20T22:12:23Z
dc.date.available	2015-07-20T22:12:23Z
dc.identifier	10.1186/1472-6920-11-43
dc.identifier	1472-6920
dc.identifier	https://doaj.org/article/db0e21ff26a549b88f7f98ff22c9e1cb
dc.identifier.uri	http://evidence.thinkportal.org/handle/123456789/15188
dc.description	<p>Abstract</p> <p>Background</p> <p>Professionalism is a difficult construct to define in medical students but aspects of this concept may be important in predicting the risk of postgraduate misconduct. For this reason attempts are being made to evaluate medical students' professionalism. This study investigated the psychometric properties of Selected Response Questions (SRQs) relating to the theme of professional conduct and ethics comparing them with two sets of control items: those testing pure knowledge of <it>anatomy</it>, and; items evaluating the ability to integrate and apply knowledge ("<it>skills</it>"). The performance of students on the SRQs was also compared with two external measures estimating aspects of professionalism in students; peer ratings of professionalism and their Conscientiousness Index, an objective measure of behaviours at medical school.</p> <p>Methods</p> <p>Item Response Theory (IRT) was used to analyse both question and student performance for SRQs relating to knowledge of professionalism, pure anatomy and skills. The relative difficulties, discrimination and 'guessabilities' of each theme of question were compared with each other using Analysis of Variance (ANOVA). Student performance on each topic was compared with the measures of conscientiousness and professionalism using parametric and non-parametric tests as appropriate. A <it>post-hoc </it>analysis of power for the IRT modelling was conducted using a Monte Carlo simulation.</p> <p>Results</p> <p><it>Professionalism </it>items were less difficult compared to the <it>anatomy </it>and <it>skills </it>SRQs, poorer at discriminating between candidates and more erratically answered when compared to <it>anatomy </it>questions. Moreover <it>professionalism </it>item performance was uncorrelated with the standardised Conscientiousness Index scores (rho = 0.009, p = 0.90). In contrast there were modest but significant correlations between standardised Conscientiousness Index scores and performance at <it>anatomy </it>items (rho = 0.20, p = 0.006) though not <it>skills </it>(rho = .11, p = .1). Likewise, students with high peer ratings for professionalism had superior performance on anatomy SRQs but not <it>professionalism </it>themed questions. A trend of borderline significance (p = .07) was observed for performance on <it>skills </it>SRQs and professionalism nomination status.</p> <p>Conclusions</p> <p>SRQs related to professionalism are likely to have relatively poor psychometric properties and lack associations with other constructs associated with undergraduate professional behaviour. The findings suggest that such questions should not be included in undergraduate examinations and may raise issues with the introduction of Situational Judgement Tests into Foundation Years selection.</p>
dc.language	English
dc.publisher	BioMed Central
dc.relation	http://www.biomedcentral.com/1472-6920/11/43
dc.relation	https://doaj.org/toc/1472-6920
dc.rights	CC BY
dc.source	BMC Medical Education, Vol 11, Iss 1, p 43 (2011)
dc.subject	Medicine (General)
dc.subject	R5-920
dc.subject	Medicine
dc.subject	R
dc.subject	DOAJ:Medicine (General)
dc.subject	DOAJ:Health Sciences
dc.subject	Special aspects of education
dc.subject	LC8-6691
dc.subject	Education
dc.subject	L
dc.subject	DOAJ:Education
dc.subject	DOAJ:Social Sciences
dc.subject	Medicine (General)
dc.subject	R5-920
dc.subject	Medicine
dc.subject	R
dc.subject	DOAJ:Medicine (General)
dc.subject	DOAJ:Health Sciences
dc.subject	Special aspects of education
dc.subject	LC8-6691
dc.subject	Education
dc.subject	L
dc.subject	DOAJ:Education
dc.subject	DOAJ:Social Sciences
dc.subject	Medicine (General)
dc.subject	R5-920
dc.subject	Medicine
dc.subject	R
dc.subject	Special aspects of education
dc.subject	LC8-6691
dc.subject	Education
dc.subject	L
dc.subject	Medicine (General)
dc.subject	R5-920
dc.subject	Medicine
dc.subject	R
dc.subject	Special aspects of education
dc.subject	LC8-6691
dc.subject	Education
dc.subject	L
dc.subject	Medicine (General)
dc.subject	R5-920
dc.subject	Medicine
dc.subject	R
dc.subject	Special aspects of education
dc.subject	LC8-6691
dc.subject	Education
dc.subject	L
dc.title	Evaluating professionalism in medical undergraduates using selected response questions: findings from an item response modelling study
dc.type	article