Semantic processing can break down in qualitatively distinct ways in different neuropsychological populations. Previous studies have shown that patients with multimodal semantic impairments following stroke - referred to as semantic aphasia (SA) - show deficits on a range of conceptual tasks due to a failure of semantic control processes in the context of prefrontal and/or temporoparietal infarction. Although a deficit of semantic control would be expected to impair performance in all modalities in parallel, most previous research in this patient group has focussed primarily on tasks employing words. This study explored the consequences of deregulated semantic cognition for an indisputably non-verbal task-naturalistic object use. Patients with SA performed more poorly than control participants on a range of everyday tasks assessed by the Naturalistic Action Test (NAT, Schwartz, M. F., Buxbaum, L. J., Ferraro, M., Veramonti, T., & Segal, M. (2002). Naturalistic action test. Thames Valley Test Company). Moreover, their scores on this assessment correlated with those obtained on language-based semantic tasks, suggesting that a common deficit could underlie the impairment in both modalities. As previously observed in the verbal domain, performance on the NAT was poorer when control processes were taxed by dual-task situations and the inclusion of semantically related distracting objects. A number of characteristics of the patients' action sequences were specifically indicative of deregulated semantic cognition. Their everyday action sequences were highly fragmented by a tendency to abandon subtasks before their completion and engage, instead, in extended periods of aimless "toying" with objects. The patient group also exhibited recurrent perseverative behaviour. These findings parallel the performance of a recurrent connectionist model of naturalistic action developed by Botvinick and Plaut [Botvinick, M. & Plaut, D. C. (2004). Doing without schema hierarchies: A recurrent connectionist approach to normal and impaired routine sequential action. Psychological Review, 111, 395-429], after the mechanism responsible for controlling action in a temporally sensitive manner was damaged. This study provides converging evidence for a failure of control processes underlying semantic memory impairment in SA, which is reflected not only in patients' performance on language-based tasks, but also in the non-verbal domain of naturalistic object use.