Research in information and communication technology in education places an increasing emphasis on the use of qualitative analysis (QA). A considerable number of approaches to QA can be adopted, but it is not always clear that researchers recognize either the differences between these approaches or the principles that underlie them. Phenomenography is often identified by researchers as the approach they have used, but little evidence is presented to allow anyone else to assess the objectivity of the results produced. This paper attempts to redress the balance. A small-scale evaluation was designed and conducted according to ‘pure' phenomenographic principles and guidelines. This study was then critiqued within the wider context of QA in general. The conclusion is that pure phenomenography has some procedural weaknesses, as well as some methodological limitations regarding the scope of the outcomes. The procedural weaknesses can be resolved by taking account of good practice in QA. The methodological issues are more serious and reduce the value of this approach for research in collaborative learning environments.