Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: a simulation and survey

dc.creator	Krebs René
dc.creator	Hofer Rainer
dc.creator	Rogausch Anja
dc.date	2010-11-01T00:00:00Z
dc.date.accessioned	2015-07-20T22:05:53Z
dc.date.available	2015-07-20T22:05:53Z
dc.identifier	10.1186/1472-6920-10-85
dc.identifier	1472-6920
dc.identifier	https://doaj.org/article/fbb57ed1ef74492e9f0503d7f9e3f187
dc.identifier.uri	http://evidence.thinkportal.org/handle/123456789/10111
dc.description	<p>Abstract</p> <p>Background</p> <p>Many medical exams use 5 options for multiple choice questions (MCQs), although the literature suggests that 3 options are optimal. Previous studies on this topic have often been based on non-medical examinations, so we sought to analyse rarely selected, 'non-functional' distractors (NF-D) in high stakes medical examinations, and their detection by item authors as well as psychometric changes resulting from a reduction in the number of options.</p> <p>Methods</p> <p>Based on Swiss Federal MCQ examinations from 2005-2007, the frequency of NF-D (selected by <1% or <5% of the candidates) was calculated. Distractors that were chosen the least or second least were identified and candidates who chose them were allocated to the remaining options using two extreme assumptions about their hypothetical behaviour: In case rarely selected distractors were eliminated, candidates could randomly choose another option - or purposively choose the correct answer, from which they had originally been distracted. In a second step, 37 experts were asked to mark the least plausible options. The consequences of a reduction from 4 to 3 or 2 distractors - based on item statistics or on the experts' ratings - with respect to difficulty, discrimination and reliability were modelled.</p> <p>Results</p> <p>About 70% of the 5-option-items had at least 1 NF-D selected by <1% of the candidates (97% for NF-Ds selected by <5%). Only a reduction to 2 distractors and assuming that candidates would switch to the correct answer in the absence of a 'non-functional' distractor led to relevant differences in reliability and difficulty (and to a lesser degree discrimination). The experts' ratings resulted in slightly greater changes compared to the statistical approach.</p> <p>Conclusions</p> <p>Based on item statistics and/or an expert panel's recommendation, the choice of a varying number of 3-4 (or partly 2) plausible distractors could be performed without marked deteriorations in psychometric characteristics.</p>
dc.language	English
dc.publisher	BioMed Central
dc.relation	http://www.biomedcentral.com/1472-6920/10/85
dc.relation	https://doaj.org/toc/1472-6920
dc.rights	CC BY
dc.source	BMC Medical Education, Vol 10, Iss 1, p 85 (2010)
dc.subject	Medicine (General)
dc.subject	R5-920
dc.subject	Medicine
dc.subject	R
dc.subject	DOAJ:Medicine (General)
dc.subject	DOAJ:Health Sciences
dc.subject	Special aspects of education
dc.subject	LC8-6691
dc.subject	Education
dc.subject	L
dc.subject	DOAJ:Education
dc.subject	DOAJ:Social Sciences
dc.subject	Medicine (General)
dc.subject	R5-920
dc.subject	Medicine
dc.subject	R
dc.subject	DOAJ:Medicine (General)
dc.subject	DOAJ:Health Sciences
dc.subject	Special aspects of education
dc.subject	LC8-6691
dc.subject	Education
dc.subject	L
dc.subject	DOAJ:Education
dc.subject	DOAJ:Social Sciences
dc.subject	Medicine (General)
dc.subject	R5-920
dc.subject	Medicine
dc.subject	R
dc.subject	Special aspects of education
dc.subject	LC8-6691
dc.subject	Education
dc.subject	L
dc.subject	Medicine (General)
dc.subject	R5-920
dc.subject	Medicine
dc.subject	R
dc.subject	Special aspects of education
dc.subject	LC8-6691
dc.subject	Education
dc.subject	L
dc.subject	Medicine (General)
dc.subject	R5-920
dc.subject	Medicine
dc.subject	R
dc.subject	Special aspects of education
dc.subject	LC8-6691
dc.subject	Education
dc.subject	L
dc.title	Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: a simulation and survey
dc.type	article