Medical education faces ever-increasing demands: larger amounts of knowledge have to be taught to larger student numbers in a time- and cost-effective manner. Therefore, new media have gained respect and raised high expectations, due to their obvious advantages. Hence, the aim of this study was to develop a standardised instrument, which allows for valid assessment of parameters relevant for the evaluation of computer-based learning (CBL). The main emphasis was placed on the overall acceptance of CBL as well as on some pertinent characteristics of individual learners. We took this as an example to systematically describe the construction of a questionnaire, focussing on the methodological and scientific aspects of this multi-step process. A first version, which was constructed on theoretical grounds, was analysed and revised using standard pretest conditions (n=18). The second revision was done using a cognitive pretest-technique, called "Think-Aloud" (n=10). After this qualitative evaluation, the questionnaire was offered to a representative sample of n=328 third-year medical students. In this quantitative analysis, 12 valid psychometric scales could be confirmed by factor analysis. Reliability analysis yielded satisfying values (a=0.71-0.91) in 10 cases: 5 scales on learner properties, 2 scales on learning strategies, one scale on the attitude towards CBL and 2 scales on expectations regarding CBL. For 2 scales (a=0.66-0.67), we identified a need for further revision in order to improve accuracy. The described questionnaire provides a valid, quantitative means to assess relevant parameters to evaluate CBL interventions.