Multisense Realism

In music, timbre also known as tone color or tone quality from psychoacoustics, is the quality of a musical note or sound or tone that distinguishes different types of sound production, such as voices and musical instruments, string instruments, wind instruments, and percussion instruments. The physical characteristics of sound that determine the perception of timbre include spectrum and envelope.

_{a signal with its envelope marked in red}

What I remember from my 7th grade general music class about timbre is that it is what made modern pop and rock different from other forms of music. All of those sound effects that become possible with electric amplification allow us to transform the same kinds of melodies that have always been a part of music into new kinds of sonic textures. Any song can be made into a heavy metal or punk song by cranking up the amplitude and distortion and manipulating the tempo.

The Wiki gives a list of subjective experiences and objective acoustic properties, such as:

_{Subjective     Objective
Vibrato           Frequency modulation
Tremolo          Amplitude modulation}

I think that this a clue that timbre can be used meaningfully as a jumping off point for understanding subjectivity and consciousness. If we think of the red signal envelope as a cross section of a 3D prismatic glacier behind it, the axis of that figurative third dimension would be where private significance (aesthetic quality) intersects public spacetime.

I’m not saying that qualia can literally be reduced to a kind of perpendicular meta-spacetime, but that it can be figuratively thought to cast a shadow which is measurable in those terms. The public universe is flat in comparison to the private universe, even though any particular private experience of eternity is of course orders of magnitude narrower than the totality of all experience. The full extent of our own lives and the lives of others is hidden by the constraint of our insensitivity. Our attention is directed to public facing events, to places and times outside of our private here and now.

Timbre at least recognizes the dipole of subjective and objective dimensions of phenomena and the sensible link between them. Our Western approach of psychoacoustics still reduces the subjective psycho- qualities to a mechanical model which is presumed to be driven by the acoustics only, but what I suggest is that acoustic mechanics are themselves micro-aesthetic experiences which are shared on all phenomenal levels (sub-personal, personal, and super-personal).

Timbre is a word which has been called “the psychoacoustician’s multidimensional waste-basket category for everything that cannot be labeled pitch or loudness”. The difference between a recording of a flute and a recording of a guitar playing the same song could be described in terms of timbre. Each instrument has its own idiosyncratic mixture of tonal and extra-tonal noise-like qualities. When we talk about playing a song on the piano, we are treating the melody as the object. What is being played is an abstract concept of sequenced notes, but what is bringing that abstraction into a concretely realized experience is the acoustic and artistic qualities of the instrument and the performance.

A font or typeface can be seen in a similar way. The invention of customizable fonts was one of the favorite features of early word processors. Instead of being locked into a particular typewriters look, the consumer could now make their printed output look more like published text. Font designers had to painstakingly build the look of each letter in the character set (since a computer would not be much good at guessing what a Helvetica style might be). Soon the entire typographic palette of styles and sizes were available, bringing into sharp distinction the difference between the alphanumeric data (ASCII text), and the font (character set). We can understand, that just as the melody of a song sounds different when it is played on a flute than when it is blasting out of a distorted electric guitar, a phrase written in Times New Roman carries a different meaning when it is written in Comic Sans. The computer, however, does not have any need to differentiate. It doesn’t care what font you write a program in, just as sheet music doesn’t carry instructions for a flute to sound like a flute.

If we use timbre and typeface as metaphors for qualia, we can see how it describes a difference in kind from the skeletal underpinnings of quanta. It’s true that we can use quantitative mechanisms to communicate qualia, but only if the receiver has the right kind of sensory range to match the sender’s intent. The mechanism has no way to appreciate the aesthetic content though, as all such qualities, whether they are fonts, timbre, image, or meaning, are reduced to the same binary digits. Compare this with what we hear, for example, when dragging a stick along the ground. There is the sound of the foot of the stick scraping the ground itself, and then there is the amplified, woody resonance which describes the length, mass, and density of the stick. Both can be felt as well as heard. The effect overall is a unified gestalt. Unlike a computation in which data must be explicitly separated and handled differently (some bits belong to the font, some to the text), natural sound is a feeling for our body and a knowing for our mind that is bound together metaphorically.

There is no mechanical process through which feelings and thoughts are bound. Instead, they are divided, like the spectrum of visible light is divided from white light, from the common sense of all experience. There is no way to simulate that coherence mechanically, but given enough artificial cues, our naturally poetic psyche will confabulate animation into it. We look at the words typed in a romantic script font and get a sense of a voice which might make the words seem pretentious or anachronistic. We hear the distorted guitar strings and feel that the song has become massively and explosively energetic. The ‘information’ which describes the underlying structure of the music or the typeface is non-poetic. It is only pixels and mathematical relations which relate to themselves and to mathematics in general, not to the experience of being alive. For being alive and aware, we need extra-mathematical qualities – the same qualities from which math itself emerges.