At the center of the joint project “Sonic Exegesis” by Ulla Rauter, Yara Mekawei and Hui Ye, which was created at the invitation of ORF musikprotokoll im steirischer herbst as part of a SHAPE+ Artist Residency, was the examination of the book “How to disappear” by the Egyptian artist and author Haytham El-Wardany, which deals with the concept of listening as a form of committed reference to the world, based on auditory perception, employed. Like Mekawei and Ye, Rauter has developed her own compositional means and systems of translation in order to distill and experience the music of the written text, the abstract sounds in spoken language, and the poetry that lies in the technology of speech synthesis.
But before we delve deeper into the subject matter, let’s take a look back at the beginnings: Even as a child, she would have liked to draw as much as she would have liked to make music, says Ulla Rauter. After many years of piano lessons, the artist turned to the electric guitar in her teens, and she had always liked to sing.
Ulla Rauter: After school, I completed a relatively relevant graphic design training at the “Graphische” (note: Vienna Federal Training and Research Institute of Graphic Arts). We also had subjects in which we could work more freely, and there I learned from the teachers that there were master classes at the University of Applied Arts Vienna in which you could work even more freely, with a wide variety of media.
Ulla Rauter decided to study in Brigitte Kowanz’s master class for cross-media art. Soon she created her first works in which she translated images into sound.
Ulla Rauter: I noticed that when I walk through the city, the facades of the buildings create a kind of film as I pass by, which at the same time has something musical about it. It’s about structure, rhythm, and height. I was interested in whether you can translate these rhythms that you see into sound. I did that digitally. I photograph the facades, and take black-and-white photos of them, one picture every ten meters. I have a pretty strict system. Everything must be exactly right, the height and also the distance. In this way, I can transfer the building into two-dimensional space and finally use the image generated in this way 1 to 1 as a digital score. It is scanned from left to right and translated pixel by pixel into sound with sine tones.
During her studies at the University of Applied Arts in Vienna, Ulla Rauter soon began to build her first sound interfaces. A happy coincidence also played a role here.
Ulla Rauter: I had a small soldering kit for a lie detector. The skin conductance could be used to control a small light, which became more and more intense with increasing nervousness. I then had another kit, a tone generator, which produced a square wave signal, a squeaky electric sound. I connected the two kits to each other on a trial basis, i.e., short-circuited the output signal of the polygraph with the tone generator. So, I was suddenly able to control the sound with my skin conductance. A friend who was visiting at the time then said: You should make an instrument out of it! That was the beginning. This first instrument happened.
This is how the Glissando instrument was finally created. A violin bow strung with copper strings, painted on the skin, produces notes and chords. The skin, with its property as a conductor of electricity, becomes a sound-shaping surface, an electrodermal interface.
Ulla Rauter: For me, the stage situation always means exposing oneself very strongly. I started playing with this state early on by really screening myself; by incorporating the physical and psychosomatic processes into the music.
The concentrated tension before the performance also flows into the music that arises when Ulla Rauter plays glissando.
Ulla Rauter: I consciously play with stage fright, which is also needed, because without nervousness there is no contact, and no electricity flows. That is, the instrument needs a minimum of nervousness, and the music reflects this excitement that the stage situation creates.
In this way, Glissando subversively subverts the virtuosity often demanded by the audience during live performances; the assumption that a musician has total control over the body and emotions. The voice as an instrument has always played a major role in Ulla Rauter’s life, and there was a lot of singing in her family, she says.
Ulla Rauter: What fascinates me about working with the voice? You can use it to convey content, but you can also concentrate on pure sound. There is something very physical about working with the voice. The sound of a voice tells a lot about the person to whom this voice belongs. In addition, the voice is something very ephemeral, it is difficult to capture and hold onto, that is, it always has something momentary. And at the same time, she has an incredibly strong presence. Our brains are also trained to perceive and process voices. You can create percussive sounds as well as melodies with your voice. For me, the voice is the most versatile instrument – and the most complex. The voice is also extremely difficult to imitate compared to other sounds, whether you try to do so with analog or digital technologies.
To be able to perform a multi-person choral piece all by yourself, Ulla Rauter created the Fingertip Vocoder.
Rauter has also developed a whole range of interfaces that allow her to distribute electronic sounds in space using hand or finger movements. Imagine one of those animations where the camera whizzes through a virtual vein, past red and white blood cells.
Ulla Rauter: I can then stop at one point in the music to enjoy the sound that is taking place there. A sound consists of many overtones. A spectral fanning takes place here. We perceive these overtone spectra as timbres. In the case of voices, these timbres help us determine who is speaking. And in my performances, I often zoom into these overtone spectra. For me, this is also a kind of fluoroscopy.
Sounds are often visualized by means of spectrograms, which have always appealed to them aesthetically, says Rauter.
Ulla Rauter: In my diploma thesis at the end of my studies at the University of Applied Arts, I 3D-milled my own voice into a very fine material. That’s when the overtones come out very nicely lamellar-like and you can see subtleties that you can’t even consciously hear. See more here.
The artist began to draw voices by hand.
Ulla Rauter: Spectrograms are a special visualization of sound, a time-frequency fanning out. You can see all the individual overtones that you hear as one and the same sound. On the one hand, these are incredibly beautiful pictures that simply interested me aesthetically, and on the other hand, these pictures contain a code. They are encrypted content, and they also contain all the paralinguistic information about the person to whom this voice belongs, about their identity. That’s why they are also called voice prints, analogous to fingerprints.
For the project “Clay on Drawing” she painted spectrograms of voice recordings with UV paint on the wall.
Ulla Rauter: This means that the room is initially empty and then these voices become visible under UV light. In this way, the room becomes an archive of voices. The whole room is full of voices, but you can’t hear them. I implemented the project “Clay on Drawing” for the first time as part of an artist residency that was linked to a psychotherapeutic center. I did a workshop there with some clients and I also conducted interviews with them. I also interviewed a therapist. Finally, I painted excerpts from these interviews on the wall in the exhibition space, of course with the consent of all participants. At the opening of the exhibition, excerpts of these were set to music again. With a camera and a computer program, these voices can be made audible again.
In her sound calligraphy, Ulla Rauter further developed this special methodology for the visualization and retranslation of sound.
Ulla Rauter: I find this drawing of voices a very poetic act. I wanted to be able to do that in the context of a live performance. However, drawing a speech sound is a very time-consuming process because these spectrograms are so detailed. So, I thought about how to simplify this process. In a research project led by Ruth Schnell, I had the opportunity to deal more intensively with this question. I then developed these voice drawings into a calligraphic sign system. As much information as possible is taken out so that only a few strokes remain. However, the brain must still be able to identify the sound as a speech sound, so much information must remain available. In my performances, I draw speech sounds and individual words, which then combine to form a text.
The sound calligraphy was created before Deep Fake was on everyone’s lips, in retrospect one could say that they already raise those questions that urgently need to be answered in a time of ever more active use of neural networks, i.e. artificial intelligence.
Ulla Rauter: What happens to the voice when it is deconstructed to such an extent and then reassembled elsewhere? Who is this being who is speaking? In the end, this is a pure media voice, but in the performance, it confidently says “I am a voice without a body” and “I am somebody, nobody”. So, this voice speaks for itself, because I let it speak. It is an example of a voice without a body. And this topic fascinates me very much, I must say. From Wolfgang von Kempelen’s speech machines to neural networks, which are already very good at cloning human voices to pretend to be someone else. And here, of course, it starts to get problematic, because the voice has always been one of the most important signs of authenticity, a biometric feature. And that’s slowly falling away now.
For “Sonic Exegesis” Ulla Rauter has developed a new vocal instrument.
Ulla Rauter: I called it a vocal arch. It is based on a simple stringed instrument, a monochord, which I have expanded electronically. With it you can play the voice like an instrument string – you can pluck, strike, or bow it. The touch characteristics are detected by a pickup and applied in real-time to the played part of the voice recording. With one hand you play the string, with the other, you move on the fretboard in the timeline of the voice.
“Sonic Exegesis” will focus on texts from the book “How to Disappear” by Egyptian artist and author Haytham El-Wardany, which deals with the concept of listening as a form of protest that can be practiced in everyday life.
Ulla Rauter: One insight I take away from the book is: As musicians, we always talk about the fact that everything is sound and in workshops, we deal with how we perceive the city or nature as a composition. It’s just like Haytham writes. When I read his book, I realized that no sound happens by chance, but all sounds happen in a certain sequence, they are related to each other. When you go out into the street, you have the feeling of entering an acoustic chaos, but in fact, this supposed chaos is based on a system of order.
Text: Susanna Niedermayr