Symptoms of inattention and hyperactivity frequently co-occur with language difficulties in both clinical and community samples. We explore the specificity and strength of these associations in a heterogeneous sample of 254 children aged 5 to 15 years identified by education and health professionals as having problems with attention, learning and/or memory. Parents/carers rated pragmatic and structural communication skills and behaviour, and children completed standardised assessments of reading, spelling, vocabulary, and phonological awareness. A single dimension of behavioural difficulties including both hyperactivity and inattention captured behaviour problems. This was strongly and negatively associated with pragmatic communication skills. There was less evidence for a relationship between behaviour and language structure: behaviour ratings were more weakly associated with the use of structural language in communication, and there were no links with direct measures of literacy. These behaviour problems and pragmatic communication difficulties co-occur in this sample, but impairments in the more formal use of language that impact on literacy and structural communication skills are tied less strongly to behavioural difficulties. One interpretation is that impairments in executive function give rise to both behavioural and social communication problems, and additional or alternative deficits in other cognitive abilities impact on the development of structural language skills.