Chowning & Music IV

With FM synthesis (see CMB: A Brief History ), John Chowning invented a wide spread synthesis technique, which was used extensively in 1980s pop music. When exploring this novel technique, he also lay the foundation for computer-based audio spatialization. With Music 10, a descendant of Max Mathews MUSIC IV, Chowning was able to create four-channel compositions with programmed movements of virtual sound sources. This object-based approach is still used in most cases today.

Algorithm

In Turenas (1972), source movements are enhanced with Doppler shift and the proper handling of the direct-to-reverb ratio:

/images/spatial/turenas_movement.png

Displaced circular movement with Doppler shift and reverberation (Chowning, 2011).

The above presented technique was used with Lissajous movements, which are characteristic for Turenas:

/images/spatial/turenas_lissajous.png

Stereo Version

References

2018

  • Christoph von Blumröder. Zur bedeutung der elektronik in karlheinz stockhausens œuvre / the significance of electronics in karlheinz stockhausen's work. Archiv für Musikwissenschaft, 75(3):166–178, 2018.
    [abstract▼] [details] [BibTeX▼]

2015

  • Martha Brech and Henrik von Coler. Aspects of space in Luigi Nono's Prometeo and the use of the Halaphon. In Martha Brech and Ralph Paland, editors, Compositions for Audible Space, Music and Sound Culture, pages 193–204. transctript, 2015.
    [details] [BibTeX▼]
  • Michael Gurevich. Interacting with Cage: realising classic electronic works with contemporary technologies. Organised Sound, 20:290–299, 12 2015. doi:10.1017/S1355771815000217.
    [details] [BibTeX▼]

2011

  • John Chowning. Turenas: the realization of a dream. In Proceedings of the 17th Journées d\rq Informatique Musicale. 2011.
    [details] [BibTeX▼]

2010

2008

  • Marco Böhlandt. “kontakte” – reflexionen naturwissenschaftlich-technischer innovationsprozesse in der frühen elektronischen musik karlheinz stockhausens (1952–1960). Berichte zur Wissenschaftsgeschichte, 31(3):226–248, 2008.
    [details] [BibTeX▼]
  • Jonas Braasch, Nils Peters, and Daniel Valente. A loudspeaker-based projection technique for spatial music applications using virtual microphone control. Computer Music Journal, 32:55–71, 09 2008.
    [details] [BibTeX▼]

Spatial Granular Synthesis

Granular synthesis - introduced in detail in the Sound Synthesis Introduction - is particularly well suited for extensions in the spatial domain. Each grain can be assigned an individual position, thus creating spatially extended textures. Granularity also increases the localization of the single events, this emphasizing the spread. Hence, various approaches have been proposed in the past and present, implemented in all common audio programming environments.

Various Approaches

Curtis Roads' book Microsound features several examples of spatial granular synthesis (Roads, 2004). Scattering - a stochastic panning of single grains - is introduced by McLeran et al. (McLeran, 2008). The approach is designed to work with stereo and 2D loudspeaker setups. The BMSwarmGranulator (Wilson, 2008) uses the Boids algorithm for spatial granular sound textures. Designed for the BEAST, the approach aims at a 'diffuse but localized sound', rather then individually perceivable grains.

Crowd Noise Synthesis

A project at TU Berlin aimed at the parametric synthesis of crown noise, more precisely the indistinct chatter (babbling) of large groups of people (Grimaldi, 2017). The project is best described on the corresponding paper or the master thesis.

References

2017

  • Grimaldi, Vincent and Böhm, Christoph and Weinzierl, Stefan and von Coler, Henrik. Parametric Synthesis of Crowd Noises in Virtual Acoustic Environments. In Proceedings of the 142nd Audio Engineering Society Convention. Audio Engineering Society, 2017.
    [details] [BibTeX▼]

2015

  • Stuart James. Spectromorphology and spatiomorphology of sound shapes: audio-rate AEP and DBAP panning of spectra. In Proceedings of the International Computer Music Conference (ICMC). 2015.
    [details] [BibTeX▼]
  • Ryan McGee. Spatial modulation synthesis. In Proceedings of the International Computer Music Conference (ICMC). 2015.
    [details] [BibTeX▼]

2009

  • Alexander Müller and Rudolf Rabenstein. Physical modeling for spatial sound synthesis. In Proceedings of the International Conference of Digital Audio Effects (DAFx). 2009.
    [details] [BibTeX▼]

2008

  • Scott Wilson. Spatial swarm granulation. In Proceedings of the International Computer Music Conference (ICMC). 2008.
    [details] [BibTeX▼]
  • David Kim-Boyle. Spectral spatialization - an overview. In Proceedings of the International Computer Music Conference (ICMC). Belfast, UK, 2008.
    [details] [BibTeX▼]

2004

  • Curtis Roads. Microsound. The MIT Press, 2004. ISBN 0262681544.
    [details] [BibTeX▼]

2002

  • David Topper, Matthew Burtner, and Stefania Serafin. Spatio-operational spectral (SOS) synthesis. In Proceedings of the International Conference of Digital Audio Effects (DAFx). Singapore, 2002.
    [details] [BibTeX▼]

Using YAML for Configuration

At some point, a program might require more arguments on startup than a command line call can manage without complications. In that case it can help to work with configuration files, which are passed to the binary when starting it. This can be done in many ways, including plain text files or markup languages like XML, Markdown, JSON or others.

The examples in this module use YAML for passing configuration data. YAML is human-readable and easy to edit, which allows quick changes to configurations.

The YamlMan Class

The YAML manager class YamlMan is used in all following examples which make use of YAML for parsing configuration parameters in adapted versions.

The Constructor

The constructor of the class is passed a file path to a YAML file as argument:

YamlMan::YamlMan(std::string filepath)

The following behavior is specific for each example, storing the values from the YAML file to member variables:

YAML::Node config = YAML::LoadFile(filepath);

// access YAML nodes:
param1 = config["param1"].as<std::string>();
param2 = config["param2"].as<int>();
param3 = config["param3"].as<double>();

Envelopes in PD

Temporal envelopes are an essential building block of sound synthesis algorithms. They are introduced in the Control Section of the CMB content:

ADSR in PD

The ADSR envelope is the most widely used temporal envelope. The PD help files contain an example, which can serve as a starting point for an ADSR object. It can also be downloaded here: ADSR

Once stored, the object can be used in custom patches to control arbitrary parameters. The following example controls the gain of a sine wave oscillator with an ADSR envelope. The adsr object has six inlets:

  • 1: the trigger

  • 2: the peak value

  • 3: the attack time (ms)

  • 4: the decay time (ms)

  • 5: the sustain amount (0...100%)

  • 6: the release time (ms)

The delay object emulates a 555 ms key press. The only outlet is an audio rate signal with the envelope:

/images/basics/pd-adsr.png

Pulse Width Modulation

Pulse Width Modulation (PWM) is a method for changing the timbre of a square wave. It is frequently found in analog and digital synthesizers as a means for enriching a sound, for example in pads.

A PWM signal can be generated with a case based logic, respectively a threshold. In programming, this can be implemented with a phasor and the pulse width $\tau$:

$$ \mathrm{PWM}(t) = \begin{cases} 1 \ \ \ \ \text{for} \ t<= \frac{T \tau}{100} , \\ -1 \ \text{for} \ t> \frac{T \tau}{100} , \end{cases} $$

The pulse width usually takes values beween $0\%$ and $100\%$:

Organizing Processes with systemd

systemd is a set of tools managing, among other things, the startup procedure on Linux systems. Within the Linux user and developer community, there is a debate whether systemd violates the Unix philosophy - however, it works well for starting all the software components we need when booting the server or the Access Points.

System services can be started in dependence on other services, This makes it possible to start a system with many interrelations. They can be started and stopped during operation. Depending on the configuration, the services can also be restarted automatically if they crash.

Creating Services

systemd services can be declared as user services or system services. They are defined in text files, which need to be located in one of the the following directories:

/usr/lib/systemd/user
/usr/lib/systemd/system

The JACK Service

The JACK service needs to be started before all other audio applications, since they rely on a running JACK server. A file jack.service defines the complete service:

[Unit]
Description=Jack audio server
After=sound.target local-fs.target

[Install]
WantedBy=multi-user.target

[Service]
Type=simple
PrivateTmp=true
Environment="JACK_NO_AUDIO_RESERVATION=1"
ExecStart=/usr/bin/jackd -P 90 -a a -d dummy -p 128
LimitRTPRIO=95
LimitRTTIME=infinity
LimitMEMLOCK=infinity
User=studio

JACK must be run as a normal user. The file above describes a system service that starts jack as user studio. But only administrators are allowed to control system services. If we want to control a service as a normal user we need a user service without the User=studio entry.

Managing Services

Once the service files are in place, several simple commands are available for controlling them. They differ, depending on whether a user service or system service is controlled. The following examples refer to the JACK user service. Controlling system services requires root privileges and do not need the --user flag.

Starting a Service

systemctl --user start jack.service

Stopping a Service

systemctl --user stop jack.service

Activating a Service

Activating a service creates a symlink in ~/.config/systemd/user/multi-user.target.wants/jack.service, pointing to the original file /usr/lib/systemd/user/jack.service. Afterwards, the system is launched after the first login of the user and stopped after the last user session exits.

systemctl --user enable jack.service

Deactivating a Service

systemctl --user disable jack.service

Getting a Service's Status

The following command prints a system's status:

systemctl --user status jack.service

When the JACK sevice has been started sucessfully, the output looks as follows:

 ● jack.service - Jack audio server
  Loaded: loaded (/usr/lib/systemd/user/jack.service; enabled; vendor preset: enabled)
  Active: active (running) since Tue 2021-04-13 23:00:14 BST; 3s ago
Main PID: 214518 (jackd)
  CGroup: /user.slice/user-1000.slice/user@1000.service/jack.service
          └─214518 /usr/bin/jackd -P 90 -a a -d dummy -p 256

Start user service on boot

Sometimes it is practical to have a user session running after the last session closes. For example if you access a server only via SSH. To achieve this we have to set the specific user to be lingering. This user's services will start at boot and quit at shutdown now.

# loginctl enable-linger studio

Organizing Processes with systemd

systemd is a set of tools managing, among other things, the startup procedure on Linux systems. Within the Linux user and developer community, there is a debate whether systemd violates the Unix philosophy - however, it works well for starting all the software components we need when booting audio servers.

System services can be started in dependence on other services, This makes it possible to start a system with many interrelations. They can be started and stopped during operation. Depending on the configuration, the services can also be restarted automatically if they crash.

Creating Services

systemd services can be declared as user services or system services. They are defined in text files, which need to be located in one of the the following directories:

/usr/lib/systemd/user
/usr/lib/systemd/system

The JACK Service

The JACK service needs to be started before all other audio applications, since they rely on a running JACK server. A file jack.service defines the complete service:

[Unit]
Description=Jack audio server
After=sound.target local-fs.target

[Install]
WantedBy=multi-user.target

[Service]
Type=simple
PrivateTmp=true
Environment="JACK_NO_AUDIO_RESERVATION=1"
ExecStart=/usr/bin/jackd -P 90 -a a -d dummy -p 128
LimitRTPRIO=95
LimitRTTIME=infinity
LimitMEMLOCK=infinity
User=studio

JACK must be run as a normal user. The file above describes a system service that starts jack as user studio. But only administrators are allowed to control system services. If we want to control a service as a normal user we need a user service without the User=studio entry.

Managing Services

Once the service files are in place, several simple commands are available for controlling them. They differ, depending on whether a user service or system service is controlled. The following examples refer to the JACK user service. Controlling system services requires root privileges and do not need the --user flag.

Starting a Service

systemctl --user start jack.service

Stopping a Service

systemctl --user stop jack.service

Activating a Service

Activating a service creates a symlink in ~/.config/systemd/user/multi-user.target.wants/jack.service, pointing to the original file /usr/lib/systemd/user/jack.service. Afterwards, the system is launched after the first login of the user and stopped after the last user session exits.

systemctl --user enable jack.service

Deactivating a Service

systemctl --user disable jack.service

Getting a Service's Status

The following command prints a system's status:

systemctl --user status jack.service

When the JACK sevice has been started sucessfully, the output looks as follows:

 ● jack.service - Jack audio server
  Loaded: loaded (/usr/lib/systemd/user/jack.service; enabled; vendor preset: enabled)
  Active: active (running) since Tue 2021-04-13 23:00:14 BST; 3s ago
Main PID: 214518 (jackd)
  CGroup: /user.slice/user-1000.slice/user@1000.service/jack.service
          └─214518 /usr/bin/jackd -P 90 -a a -d dummy -p 256

Start user service on boot

Sometimes it is practical to have a user session running after the last session closes. For example if you access a server only via SSH. To achieve this we have to set the specific user to be lingering. This user's services will start at boot and quit at shutdown now.

# loginctl enable-linger studio

Wavefolding

Wavefolding

Wavefolding is a special case of waveshaping, working with periodic transfer functions. Depending on the pre-gain, the source signal gets folded back, once a maximum of the transfer function is reached. Compared to the previously introduced soft clipping or other methods of waveshaping, this adds many strong harmonics.

Periodic Shaping Function

A simple basic transfer function is a sine with the appropriate scaling factor. The pre-gain $g$ is the parameter for controling the intensity of the folding effect:

$$ y[n] = sin( g \frac{\pi}{2} x[n]) $$

For an input signal $x$, limited to values between $-1$ and $1$, respectively for gains $g\leq1$, this results in a sinusoidal waveshaping function with saturation:

Text(0,0.5,'y')

When the input signal exceeds the boundaries $-1$ and $1$, the signal does not clip but is folded back. This can be achieved by amplifying the input with an additional gain:

For a gain of $g=3$, the time-domain output signal looks as follows:

Spectrum for a Sinusoidal Input

The spectrum of wavefolding can be calculated by expressing the folding term as a Fourier series: The Jacobi–Anger expansion can be used for this purpose, with the pre-gain $g$:

$$ \sin(g \sin(x)) = 2 \sum\limits_{m=1}^{\infty} J_{2m-1}(g) \sin((2m-1)x) $$

At this point it is already apparent that the resulting signal contains harmonics at odd integer multiples of the fundamental frequency $f_m = 100 \mathrm{Hz}\ (2 m -1)$. Their gain is determined by first kind Bessel functions $J_{2m-1}(g)$:

For the DFT this leads to:

$$ \begin{eqnarray} X[k] &=& 2 \sum\limits_{m=1}^{\infty} J_{2m-1}(g) \sin((2m-1)x) \sum\limits_{n=0}^{N-1} e^{-j 2 \pi k \frac{n}{N}} \\ % % % X[k] &=& 2 \sum\limits_{n=0}^{N-1} \sum\limits_{m=1}^{\infty} J_{2m-1}(g) \sin((2m-1)x)\ e^{-j 2 \pi k \frac{n}{N}} \\ % % % X[k] &=& 2 \sum\limits_{n=0}^{N-1} \sum\limits_{m=1}^{\infty} J_{2m-1}(g) \frac{1}{2} \left( e^{j (2m-1)x} - e^{-j(2m-1)x} \right) % \sin((2n-1)x)\ e^{-j 2 \pi k \frac{n}{N}} \\ % % % X[k] &=& \sum\limits_{n=0}^{N-1} \sum\limits_{m=1}^{\infty} J_{2m-1}(g) \left( e^{-j 2 \pi k \frac{n}{N} + j (2m-1)x } - e^{-j 2 \pi k \frac{n}{N} -j(2m-1)x} \right) % \sin((2n-1)x)\ \end{eqnarray} $$

With $x = 2 \pi \frac{f_0}{f_s} n$

$$ X[k] = \sum\limits_{n=0}^{N-1} \sum\limits_{m=1}^{\infty} J_{2m-1}(g) \left( e^{-j 2 \pi k \frac{n}{N} + j (2m-1) 2 \pi \frac{f_0}{f_s} } - e^{-j 2 \pi k \frac{n}{N} -j(2m-1) 2 \pi \frac{f_0}{f_s}} \right) % \sin((2n-1)x)\ $$

1 Hints on this by Peyam Tabrizian can be found here: https://youtu.be/C641y-z3aI0

DFT Plots

The below plots show the spectra of the folding operation for a sine input of $100 \mathrm{Hz}$ at different gains. With increasing gain, partials are added at the odd integer multiples of the fundamental frequency $f_m = 100 \mathrm{Hz}\ (2 m -1)$:

[ 100  300  500  700  900 1100 1300 1500 1700 1900] ...

Combining Nodes in SuperCollider

Creating and Connecting Nodes

Audio buses can be used to connect synth nodes. In this example we will create two nodes - one for generating a sound and one for processing it. First thing is an audio bus:

~aBus = Bus.audio(s,1);

The ~osc node generates a sawtooth signal and the output is routed to the audio bus:

~osc = {arg out=1; Out.ar(out,Saw.ar())}.play;

~osc.set(\out,~aBus.index);

The second node is a simple filter. Its input is set to the index of the audio bus:

~lpf = {arg in=0; Out.ar(0, LPF.ar(In.ar(in),100))}.play;

~lpf.set(\in,~aBus.index);

Moving Nodes

/images/basics/sc-order-1.png

Node Tree before moving the processor node.


The moveAfter() function is a quick way for moving a node directly after a node specified as the argument. The target node can be either referred to by its node index or by the related name in sclang:

~lpf.moveAfter(~osc)

/images/basics/sc-order-2.png

Node Tree after moving the processor node.

More APIs

There are many more APIs which can be used for real time or off line sonification. Several projects and meta sites list examples by category:

NASA

NASA offers a great variety of open APIs with data from astronomy: https://api.nasa.gov/

WHO

https://www.who.int/data/gho/info/gho-odata-api

Create an API

https://towardsdatascience.com/a-layman-guide-for-data-scientists-to-create-apis-in-minutes-31e6f451cd2f