Henningsen-Schomers, M.; Garagnani, M. and Pulvermüller, F.. 2022. 'Influence of language on concept formation and perception in a brain-constrained deep neural network model'. In: TABU Dag 2022. Groningen, Netherlands 9 - 22 June 2022. [Conference or Workshop Item]

Preview

Text TABU_2022_paper_23.pdf - Submitted Version Available under License Creative Commons Attribution Non-commercial. Download (396kB) | Preview

Abstract or Description

Whether language influences perception and thought remains a subject of intense debate (1, 2). We address this question in a brain-constrained neurocomputational model (3) of fronto-occipital (extrasylvian) and fronto-temporal (perisylvian) cortex including spiking neurons. The unsupervised neural network was simultaneously presented with word forms (phonological patterns, “labels”) in perisylvian areas and semantic grounding information (sensory-motor patterns, “percepts”) in extrasylvian areas representing either concrete or abstract concepts. Following the approach used in a previous simulation (4), each to-be-learned concept was modeled as a triplet of partly overlapping percepts; the models were trained under two conditions: each instance of a perceptual triplet (patterns in extrasylvian areas) was repeatedly paired with patterns in perisylvian areas consisting of either (a) a corresponding word form (label condition), or (b) noise (no-label condition). We quantified the emergence of neuronal representations for the conceptually-related percepts using dissimilarity (Euclidean distance) of neuronal activation vectors during perceptual stimulation. Category learning was measured as the difference between within- and between concept dissimilarity values (DissimDiff) of perceptual activation patterns.

A repeated-measures ANOVA with factors SemanticType (concrete/abstract) and Labelling showed main effects of both SemanticType and Label, and a significant interaction. We also quantified the “label effect” in percentage change from NoLabel to Label conditions, separately for between- and within-category dissimilarities. This showed that the label effect was mainly driven by changes in between-category dissimilarity, was significantly larger for abstract than concrete concepts, and became even larger in the “deeper” layers of the model.

Providing a referential verbal label during the acquisition of a new concept significantly improves the cortex’ ability to develop distinct semantic-category representations from partly-overlapping (and non-overlapping) perceptual instances. Crucially, this effect is most pronounced in higher order semantic-hub areas of the network. These results provide the first neurocomputational evidence for a “Whorfian” effect of language on perception and concept formation.

Item Type:	Conference or Workshop Item (Talk)
Departments, Centres and Research Units:	Computing
Dates:	Date Event 6 March 2022 Submitted 22 March 2022 Accepted 9 June 2022 Completed
Event Location:	Groningen, Netherlands
Date range:	9 - 22 June 2022
Item ID:	31555
Date Deposited:	07 Mar 2022 13:35
Last Modified:	25 Jun 2022 16:38
URI:	https://research.gold.ac.uk/id/eprint/31555

View statistics for this item...

Edit Record (login required)