Synaesthesia

“‘The Underlying Gesture’. Towards the Notion of Gesture in Jean d’Udine and Sergei Eisenstein”. In: Rossella Catanese, Francesca Scotto Lavina, Valentina Valente (eds.), From Sensation to Synaesthesia in Film and New Media. Cambridge Scholars Publishing 2019, 102-115.

Synaesthetic metaphors and sound
This paper analyses a phenomenon called synaesthetic metaphors, in which one sensory modality is described in terms of another (e.g. bright sound, sharp taste, sweet smell). The aim of the research, based on the contrastive analysis of Slovenian and French music reviews, is to determine which sense domains are employed for (synaesthetic) description of sound, more precisely music, and how frequent they are. In this regard the paper examines the directionality principle according to which more concrete and more accessible lower senses (touch, taste, smell) tend to map onto less concrete and less accessible higher senses (sound, sight). Furthermore, the analysis focuses on two other aspects: the conventionality of synaesthetic metaphors and their connotative meaning. Since the source and target domain in synaesthetic transfers belong to sense domains, the paper emphasize the experiential – i.e. perceptive, cognitive, and sociocultural – dimension of this phenomenon, with regard to which rises the main question concerning the universal and cultural tendencies of synaesthetic metaphors.

Key words: synaesthesia, metaphor, perception, music, sound, cognitive linguistics

Within the strategies built upon the intentional weakening of the diegetic efficiency a group apart is formed by those films which try to reach this gesture of destruction by resorting to the diegetically weaker, in this sense pre-discursive way of expression of the silent cinema. In this paper the author analyze two films from this perspective: Guy Maddin’s Careful (1992) and Esteban Sapir’s La Antena (2007).
The plot of these films coupled with the effect achieved by the mimicry of the silent film challenges interpretation. What are the discursive functions in the contemporary cinema of the silent film-like formal principles and techniques weakening the diegetic effect, such as the absence of the synchronized sound and the use of the intertitles in the film? The hypothesis is, that the turning to the visual style of the silent film is associated with the problematization of the medial relationship between image and text, image and voice respectively, and is manifested both at the level of the fable and in the use of the filmic techniques.

Synesthesia is a perceptual phenomenon involving the stimulation of multiple senses. In this work, we determine the presence of sound-gustatory synesthesia by looking at the possible effects of background music on the perceived taste of a coffee-sugar mixture. We asked participants (N = 83) to listen to music while identifying the tastes they perceived drinking a coffee-sugar sample. Our results showed that sweetness was perceived more while listening to the "Slow" music, which is consistent with previous work. The perception of sourness also increased with the tempo of the music, consistent with work associating sourness with pitch. Interestingly, participants also perceived saltiness and sourness even though the ingredients did not contain ingredients with those tastes, which provides further evidence of sound influencing taste perception. This study has shown the presence of sound-gustatory synesthesia in a typical coffeehouse setting, introducing potential applications in psychophysics, food science, and other complex systems research. Our algorithm has also shown how quantitative tools can be used in a qualitative field such as psychological perception. We expect multisensory, interconnected technology in the Internet of Things to spread the experience of synesthesia within a population, with Big Data enabling researchers to detect and measure synesthesia much more accurately.

McKenna, D. (2022, January). Psychedelics and evolution: The stoned ape theory . Invited talk for the Psychedelicacies Salon at the Viktor Wynd's Museum of Curiosities & Last Tuesday Society (online), 30th January. Hosted and curated by David Luke & Maya Bracknell Watson.

https://www.thelasttuesdaysociety.org/event/dennis-mckenna-on-psychedelics-and-evolution-the-stoned-ape-theory/

"Dalla IV di copertina

Sentire il design significa cogliere sensazioni con l'orecchio, ma anche tastare, odorare, vedere, gustare il progetto. L'abilità del progettista visivo sta certo nel far vedere, nel definire gerarchie percettive fra scritture e figure, ma anche nella capacità di orchestrare i registri sensoriali e di scegliere cosa lasciar catturare all'occhio e cosa invece agli altri sensi. Ciò non vale solo per artefatti pluriregistro come i multimedia; anche un libro comporta un coinvolgimento sensoriale plurimo, ha una tattilità, un peso, un odore, produce sonorità nell'azione dello sfogliare. Il volume mostra l'importanza, i caratteri e i modi di una progettazione mirata alla totalità del sensorio. Affronta gli aspetti fondamentali che legano sinestesia, comunicazione visiva e formazione del designer, delineando le conoscenze scientifiche sull'interazione sensoriale e sulla sinestesia ed esemplificando manifestazioni e casi propri della scrittura e, più in generale, del design della comunicazione. L'autrice rilegge alcuni esempi storici della grafica - dalla tipografia futurista alle esperienze formative del Bauhaus, fino ai recenti sviluppi indotti dal digitale con artefatti cinetici e interattivi - cercando ogni volta di ricostruire le relazioni fra gli intrecci fenomenici e i corrispettivi linguistici, tra le sensazioni e le parole. "

The aim of this work is to highlight the importance of the body in the design project, as a body capable to feel and reacts with pleasure or suffering during the interaction with objects and environment all around.
Basic Design, Synaesthesia and Multimedia are the three main themes of the thesis and they are analysed also by the point of view of their historic evolution. The integration of these concepts lead to the idea of a Laboratory where it is possible to make sensory experimentations with a basic design approach to understand how our senses react in front of particular visual, tactiles and sound stimuli and to develop products which are more suitable to real sensory human needs and behaviours.
--------------------------------------------------------------------------------------------------------------------------------------L’obiettivo di questo lavoro è di evidenziare l’importanza del corpo nel progetto, un corpo inteso come essere recettivo che sente, prova emozioni e reagisce ad esse con piacere o sofferenza.
Dopo una prima fase di studio sul tema del Basic Design, delle Sinestesie e della multimedialità, affrontati anche dal punto di vista della loro evoluzione storica nel tempo, si è approdati all’idea di un Laboratorio sensoriale nel quale poter effettuare sperimentazioni.
Il Basic Design, ha come finalità lo studio di una metodologia progettuale; esso è legato alla didattica del progetto ed è una disciplina che forma le basi del fare progettuale. Inoltre rappresenta l’attitudine a un certo modo di ragionare che viene poi applicato anche a progetti concreti e si occupa di temi formali sia bidimensionali che tridimensionali affrontando la complessa sfera della percezione. Tutto ciò che ci circonda infatti, è filtrato dal nostro modo di percepire che a volte è illusorio, in particolare quando stimoli di diversa natura si intrecciano e si influenzano a vicenda, come nel caso delle sinestesie.
Da tutte queste considerazioni nasce la proposta di un Laboratorio sensoriale inteso come luogo nel quale sperimentare i meccanismi di interazione fra i sensi in specifici contesti e sviluppare un esperimento sulle sinestesie, centrato su alcuni test effettuati su due gruppi di persone. I registri sensoriali coinvolti sono suono, vista e tatto. Segue poi l’idea di simulare una condizione sperimentale, come verifica del lavoro precedentemente impostato, utilizzando una telecamera e coinvolgendo un gruppo di 8 persone (4 più sensibili all’immagine e 4 più sensibili al suono) in un esperimento composto da 3 Test di cui due centrati sul rapporto suono-immagine e uno sul rapporto suono-materiale.

"Dalla IV di copertina

Capita, talvolta, di percepire sensazioni multiple o, meglio, contemporanee, come per esempio un suono o un gusto che ha una valenza di colore, un azzurro alla voce di un flauto, uno scarlatto con un sapore aspro, quasi che i nostri sensi non siano solo cinque, ma esista anche una capacità di percezione sensoriale simultanea, quella che viene definita sinestesia. Questa realtà, indagata sin dai tempi antichi, viene esplorata dall'autrice in una prospettiva progettuale, partendo dal fatto che il nostro rapporto con gli oggetti d'uso quotidiano sottende il coinvolgimento dei sensi nella loro singola specificità, ma ancor più esalta i naturali meccanismi d'interdipendenza presenti fra i sistemi sensoriali dell'uomo.
Gli artefatti, gli ambienti, sono da intendere non solo come forme che assolvono a determinate funzioni, ma anche - e primariamente - come fattori che provocano un aggregato di percezioni.
Il libro è idealmente distinto in due parti: la prima è rivolta all'analisi dell'interiorità delle sensazioni e delle loro modalità d'interagire in presenza di percezioni simultanee, con il coinvolgimento di saperi presi a prestito dalla filosofia, dalla medicina, in particolare dalla neurologia, e dalla psicologia della percezione; la seconda è rivolta all'esame dell'esteriorità dei dati sensibili - suoni, colori, forme, animazioni - e dei loro modi di essere rappresentati, con esemplificazioni di pratiche sinestesiche tratte dalla letteratura, dalle arti visive e musicali, dalla storia degli oggetti e dai prodotti della multimedialità.
Se il lavoro dei progettisti si deve confrontare, e sempre più intensamente, con la manipolazione di dati sensoriali (visivi, ma anche tattili, propriocettivi, vestibolari e uditivi), Sinestesie per il design risponde all'istanza di una guida nella caotica presenza di stimoli di questa nostra epoca multimediale."

A forced choice paradigm was employed, similar to that of a recent experiment conducted by Hubbard (1996) using paired greyscale patches and pitch intervals, which provided evidence of a universal mapping of lightness value relative to pitch height. In the current experiment we used colour, instead of greyscale, and tonal intervals within the normal vocal and instrumental midrange. 40 participants chose between colour combinations derived from two models of pitchcolour correspondence in response to intervals and dyads from all 48 major and minor keys in the cycle of 5ths. Analyses revealed a tendency to map adjacent colours onto close intervals and relate spectrally distant colours to larger intervals, particularly amongst those with musical training, but also among participants with training in visual arts. This experiment provides evidence of a perceptual analogy between pitch and colour relationships, as well as some support for the traditional linear model of pitch-colour correspondence.

This paper focuses on synaesthetic shift occurring in translation of Emily Dickinson's poetry into Ukrainian and Russian. The research is in line with Redka's (2009) view of verbal and poetic synaesthesia, a trope which is structurally represented as a word combination, a sentence or even a poem, and manifests itself in the text as the author's perception of objective reality via visual, colour, tactile, olfactory, auditory and gustatory sensation. We aim to describe two types of poetic synaesthesia: metaphoric, which is realized in the text as an image and is represented in cognition as conceptual metaphor or metonymy; and non-metaphoric, which is triggered by a combination of verbal images and phonetic instrumentation and versification. We thus hypothesise that synaesthetic shift between source and target images leads to the change of the original image and results from changes in versification and phonetic instrumenta-tion patterns in poetry, verbal images and conceptual cross-domain mapping.

Una riflessione sull’impatto culturale delle tecnologie nell’esistenza dell’uomo e del Pianeta, sull’evoluzione dello statuto del corpo, nella duplice accezione di organismo biologico e culturale. Sullo sfondo della fittizia dicotomia tra «naturale» e «artificiale», e nei limiti di una concezione antropocentrica dell’artificiale.
Nella seconda parte il filo conduttore è quello del virtuale, relativamente ai linguaggi neotecnologici di simulazione (computer image, olografia, realtà virtuale). Simulazione che concerne anche l’olfatto e il tatto, per un approccio sinestetico, polisensoriale, interdisciplinare, nei confronti di una rappresentazione che si fa sempre più globale e pervasiva. Nella seconda parte c’è anche un capitolo dedicato all’arte, perché l’artista è uno dei più efficaci, profondi e consapevoli sperimentatori dei sistemi e dei linguaggi del simbolico, contribuendo a incrementarne le potenzialità, ed è tra i pochi soggetti in grado di considerarli anche dal punto di vista etico.

Building on the one hand on the link between immersion and peripheral vision, and on the other hand on the visually induced perceptual illusion of self-motion (vection), the author examines synesthesia through the relationship between peripheral vision and proprioception. The author maintains that immersion in installations results from multimodal perception grounded in motor-sensory activity, and he considers installations and scenographies in which the viewer’s peripheral vision, balance, motion and posture play an important role.

Synästhesie als ästhetisches Phänomen verweist zunächst auf eine Durchdringung der ver-schiedenen Sinne. Die meist in einem Modell isolierter Wahrnehmung untersuchten Sinne geraten hierbei in einen Austausch, wobei sich häufig die Frage nach der Möglichkeit eines aktiven Erlernens jener Fähigkeiten stellt.

Im Mittelpunkt dieses Bandes steht die Frage nach einer in diversen Medien ermöglichten intermodalen Erfahrung. Dieser Punkt ist für den Film, der als multisensorisches Medium immer auch multimodale Wahrnehmung produziert, von grundlegender Bedeutung.
Es geht darum, das Kino und andere Medien als besondere Räume, als Schauplätze zu erkennen, an denen die sogenannte metaphorische Synästhesie ausgebildet wird.

In this paper I will describe and present examples of my live audio-visual work for 3D spatial environments. These projects use motion-tracking technology to enable users to interact with sound, light and video using their body movements in 3D space. Specific video examples of one past project (Virtual DJ) and one current project (Virtual VJ) will be shown to illustrate how flexible user interaction is enabled through a complex and precise mapping of 3D space to media control. In these projects audience members can interact with sound, light and video in real-time by simply moving around in space with a tracker in hand. Changes in sound can be synchronized with changes in light and/or real-time visual effects (i.e. music volume = light brightness = video opacity). These changes can be dynamically mapped in real-time to allow the user to consolidate the roles of DJ, VJ and light designer in one interface. This interaction model attempts to reproduce the effect of synesthesia, in which certain people experience light or color in response to music.

In 1911, Ricciotto Canudo labeled the cinema as the Seventh Art and claimed that it was superior to the other so-far existing art forms. In 2012, the British filmmaker Peter Greenaway directed yet another film that makes use of all these arts and media to convey an authorial discourse on the importance of cinema and its versatility: "Goltzius and the Pelican Company" (UK). I contend that this art house film, unemotional, filled with distancing effects and exhibiting nakedness throughout, is, in fact, an extremely sensorial piece of cinema where corporeality is not only the form but also the message, both literally and metaphorically. I advocate that this cerebral film can be sensual and sensuous, both through the use of the characters' bodies and the materiality of the text. I also consider that this film generates two types of qualified immersion in the viewer: an artistic appreciation derived from coenasthesia and film textures; and a narrative appreciation caused by alignment with the characters , their non-psychological nature notwithstanding. Allegory, as both a structural device and a conveyor of meaning, is responsible for the combination of spectatorial detachment and immersion.

f synesthesia is rare in "Le Spleen de Paris", it is also differently constructed and shaped than in the Fleurs du Mal. This article aims to demonstrate that this evolution points out a shift in Baudelaire's esthetics, that from Unity of Univers seems to move towards fragmentation of reality: actually, from Baudelaire to Proust.

Cet article s'intéresse à la synesthésie comme phénomène rhétorique et comme instance esthétique dans le Spleen de Paris de Baudelaire. Nous but est de démontrer que la rareté du phénomène et sa différente nature par rapport aux Fleurs du Mal marque une bascule dans l'esthétique baudelairienne, qui évolue de la fusion d'un univers unitaire à la fragmentation d'un monde en mouvement

The Kiki-Bouba effect comprises a relation between two abstract figures and two non-words: the star-shaped figure is called 'Kiki' and the rounded figure 'Bouba'. The effect is explained by a sound-vision synaesthesia: certain sounds are associated with certain shapes in a non-arbitrary manner.When we asked the participants to decide which of the two figures, the star-shaped or the rounded one, to call yin and which yang, some 85% choose the star-shaped figure as yin. There are previous cases of synaesthesia where personality is attributed to numbers or letters. In our results, the word Kiki is overall happy, clever, small, thin, young, unpleasant, and nervous. The starshaped figure is overall clever, tall, small, slim, nervous, unpleasant, and upper-class. That is, the correspondence above all concerns the qualifying adjectives clever, unpleasant, and nervous, as well as the physical appearance small and thin. This brings us to the fat-thin effect. Cinema, literature, comics, and children's programmes are full of contrasting figures: Don Quixote and Sancho Panza, Laurel and Hardy (called the fat man and the skinny man in Spain), Asterix and Obelix, Tintin and Captain Haddock, Bert and Ernie (Epi and Blas in Spanish), or the Spanish comic about very naughty twin boys called Zipi (with fair hair) and Zape (with dark hair). Our main conclusion is that first names and last names are not entirely arbitrary. There is a correspondence between (rounded vs. angular) names and physical characteristics (fat vs. thin objects or persons) and concepts (foolish vs. intelligent, nice vs. unpleasant). The Kiki-Bouba effect is a semantic one.

In this article, the rapidly growing body of research that has been published recently on the topic of crossmodal correspondences that involve auditory and gustatory stimuli is critically reviewed. The evidence demonstrates that people reliably match different tastes/flavours to auditory stimuli varying in both their psychoacoustic (e.g., pitch) and musical (e.g., timbre) properties. In order to stimulate progress in this relatively young research field, the present article aims at consolidating prior findings concerning specific auditory gustatory mappings, whereby special attention is given to highlighting a) any conflicts in the existing experimental evidence, and b) any potential caveats with regard to the most appropriate interpretation of prior studies. Next, potential mechanisms underlying auditory-gustatory crossmodal correspondences are discussed. Finally, a number of potentially fruitful avenues for future research are outlined.

The dissertation was written in 1997-1998 for my Master's degree when I was a student of poetry at the Sorbonne University in Paris.

It was the first step of a philosophical research, which continued later with cognitive psychology and philosophy of mind.

I had the intuition that synaesthesia was a key phenomena to understand the mind. I also considered that it would be relevant to start with an in-depth study of sensations.

Metaphor concentrates meaning and I therefore consider it interesting as an intuitive "modelisation tool" to start with.

I then had the ambition to introduce cognitive poetics in France (but this ambition came perhaps too early).

My following post-graduate (DEA) dissertation I wrote in 1999-2000 was therefore named "Éléments d'une poétique cognitive" ("Elements of cognitive poetics").Mémoire de poésie anglaise.         

----               


"Shelley was willing to “awaken sensations like those which animate[d his] bosom” that is to say awaken experiences of mental, physical and emotional feelings resembling his own and existing in their potentiality within the receptive reader’s combinational or suggestible mind. That sensory and emotional picturing should partake in the understanding and “feeling” of ideas is an evidence which was interesting to study on the limited field of the mysterious semantic and psycho-physiological complexity of Shelley’s poetic synaesthesia. 

With this thrilling aim in view, we will stick to Shelley’s personality and mentality, discovered through his essays and letters, but also rely on old and recent studies in psychology, physiology, cognition1 or symbolism, made by philosophers, psychologists, literary critics and by poets and artists. In  the first part of our dissertation which we entitled “Shelley and synaesthesia”, we will try to find a psychophysiological explanation for Shelley’s synaesthesia and in a second part, we will apply our discoveries and reflections to some chosen poems in order to evaluate the importance of this type of imagery in Shelley’s poetry.  This study has also the ambition to be a reflection on the metamorphosis of human sensibility, without which no artistic works (its avatars) can live, last, and above all be shared. We hope to cast some interesting subjective light on the secret combining qualities of the creative poetic mind. 

In other words, this dissertation  was also written as an attempt to explore the sources of art and creation. "
(1997-1998)

This keynote paper for Workshops on Psychoanalysis & Art History raises questions about the ability of neuroscientific theory to replace older psychoanalytic models. On the one hand, scholars in the arts and humanities have raised important doubts about the ability of brain science to offer real insights into such complex cultural phenomena as the history of art. If neuroscience turns out only to offer very limited theoretical tools, then it is likely we are not going to abandon psychoanalysis any time soon. On the other hand, a growing number of art historians, especially new researchers, find neuroscience more robust and ‘of the moment’ than psychoanalysis and are beginning to apply neuroaesthetic ideas to art-historical problems. Concepts such as ‘mirror neurons’ and ‘neuroplasticity’, for example, suggest new ways of understanding behavioural responses to works of art and the importance to artists of the natural environment. This paper, however, urges caution. It is argued that rather than abandoning psychoanalytic approaches to the history of art we should instead review the method. The greatest value of the current excitement about neuroscientific ‘discoveries’ might be to make us question what actually constitutes a psychoanalytic approach and to update the concepts that we all too habitually employ.

Este libro quiere proporcionar una imagen actualizada de los  conocimientos adquiridos en dos siglos de estudio, desde  comienzos del siglo XIX hasta hoy, sobre la sinestesia. Está  dirigido a los estudiosos, a los estudiantes de universidad y a conservatorios, a los que, incluso en contextos disciplinares diferentes, quieren descubrir los orígenes históricos de la sinestesia, las explicaciones neuroscientíficas, o experimentar con sus posibles aplicaciones en el lenguaje, las artes y el diseño.
El libro es el resultado de la colaboración internacional de un acreditado grupo de estudiosos que han participado en al menos en uno de los Congresos de Sinestesia: Ciencias y Arte (2005) 2007, 2009, 2012, organizado por la Fundación Internacional Artecittà, il Politecnico di Milano, la Universidad de Granada, y Universidad de Almería.

The ability to select and apply type sensitively is an art form that requires an astute knowledge of the communicative complexity of letterforms. However, as a designer and design educator, I have observed that many designers frequently select and apply typefaces inappropriately or arbitrarily because they are simply unaware of the complex meanings underlying letterforms, as well as the power of the communicative choices they make. Many designers with even a basic understanding of type still tend to prefer to use illustration, illustrative graphics, icons or photography as their primary media of communication. In the event that type is indeed used, designers tend to choose ‘clean’ typefaces because they appear to detract less from the communicative aspects of other rhetorical texts already at play in their design. In other instances, letterforms may be chosen to achieve an array of elaborately intricate design layouts that are often superficially strewn across decorative, trite and eye-candied designs.

From these observations, I have therefore dedicated my study to delineating and discussing two default modes or methods used for selecting and applying type – type as experiential form and type as iconic form – in order to illustrate the powerful, yet intricate communicative facets of the letterform. The first mode relates to how designers select type based on a typeface’s experiential form. By this, I refer to the connotations that we derive from our physical and sensual perceptions of letterform shapes. I refer to George Laukoff’s experiential metaphor theory, as well as sound-image symbolism theory (synesthesia) in order to identify reminiscent and intuitive letterform perception. The second describes several ways in which designers invoke symbolic connotation by selecting iconic typefaces. Here, I investigate historical and cultural narratives woven into iconic typefaces and how these narratives may be signified, resignified and repurposed.

As a means of understanding the interconnected nature of meaning embodied by the letterform, my final objective is to highlight letterform communication from a visual rhetorical perspective. By conducting an in-depth case study of the Fraktur typeface (as communicating at once
experientially and symbolically), I stress several tensions that exist as a result of overlapping meaning and the interconnected nature of the two default modes of type selection. I thereby argue that designers need be aware of the communicative implications of their default modes or strategies to typeface selection. My point of departure is that a more holistic approach to selecting and applying typefaces could be followed and that rhetorical theory may provide an analytical framework for such an inclusive perspective. I maintain that if the communicative
complexity of letterforms is viewed from a visual rhetorical perspective, where rhetorical intricacies of meaning embodied in the letterform are thoughtfully and holistically considered (where designers may question their default modes of type selection), designers can be more strategic in directing meaning through type.

Luke, D. (2015, July). Psychedelic synaesthesia: New light on an old phenomenon. Paper presented at Breaking Convention: The 3rd International Conference on Psychedelic Consciousness, University of  Greenwich, London, 10-12th July.

Video here: https://vimeo.com/141751685

The neurobiology of synaesthesia is receiving growing attention in the search for insights into consciousness, such as the binding problem. The cognitive processes and phenomenology of congenital synaesthesia have been much explored, yet virtually nothing exists concerning the phenomenological topography of psychedelically-induced experiences, despite the recent renaissance in psychedelic research. Results are presented of an online survey into the prevalence, type and frequency of psychedelic synaesthesia experiences along with accounts of three people reporting permanent psychedelically-induced synaesthesia – a phenomenon as yet unreported in the literature but which may have important implications for our neuroscientific understanding of the congenital condition.