Sensor-Based Projects

BinBong

The first version of the BinBong (von Coler et al, 2017) is a haptic musical interface, using a MIDI over USB wired communication. A Teensy board is used to capture the force applied to four valve-like mechanics and the wooden top. The goal of this research-oriented instrument is to investigate the expressive means of force-based control in monophonic electronic musical instruments.

BinBong MKII

Based on the findings of the BinBong, a second, wireless version was designed, using a RedBear Duo board with integrated WiFi communication (Schmied, 2018). It has been designed to control spatial sound synthesis (von Coler et al., 2020) with an additional IMU and is used within user studies on mapping and expressivity (von Coler, 2021).

GameTrak: The Gallows

The GameTrak is a motion controller for the Playstation, launched in 2004. Nowadays, it is obsolete as a gaming device and used devices are cheap. The actual control unit is basically a joystick with a string attached, allowing to measure displacement and lengthening via potentiometers:

The GameTrak is easily hacked in many ways and has thus been used in many experimental music projects and by laptop orchestras (Huberth & Nanou, 2016). It can be easily integrated into projects using PD, SC and other environments, while the potentiometers are very reliable. The follwing short demo video shows a small version of The Gallows with a simple PD sample synthesis patch:

Visit the Gallows Website for more information and audio recordings or get the code: https://github.com/anwaldt/The_Gallows

PS_Gloves

The PS_Gloves are a project by Paul Schuladen, developed between 2018 and 2023 and now being finalized in his master's thesis. They track posture and orientation of the hand, using two inertial measurement units (IMU) and various capacitive sensors at the finger tips. Within a final study, they are used to play physical models of string instruments.

Actuator-Based Projects

Symphonie pour un homme seul

With Symphonie pour un homme seul (1949–1950), Pierre Schaeffer and Pierre Henry took the first approaches to Musique Concrète to a new level, beyond the scope of simple studies. Although not a multichannel composition, the artificial reverberation used in this piece can be considered a spatial audio production technique [1]:

Pupitre d'espace

The 'pupitre d'espace' was an electromagnetic interface for live diffusion of Musique Concrète.

Installing SC-HOA

The SC-HOA library is shipped as a so called Quark and it can be installed from inside SC. Besides a GUI-based way, a single command is enough to install the complete library with all objects and classes in the system's directories:

Quarks.install("https://github.com/florian-grond/SC-HOA")

Make sure to reboot the interpreter after installing the Quark. The external classes need to be compiled.

To find out where SC has installed your external, run:

Platform.userExtensionDir

OSI Model

The OSI Model groups different services, functions and applications of telecommunication systems into seven hierarchically arranged layers:

Network based audio systems can be based on different layers. This affects their capabilities and application areas. A comprehensive list can be found here: comparison on Wikipedia

Directories

You can maneuver through the system's directories using the cd command. Changing to an absolute path /foo/bar can be done like this:

$ cd /foo/bar

New directories are created with the mkdir command. For creating a new directory mydir in your recent location, type:

$ mkdir mydir

The content of the recent directory can be listed with the ls command. The following arguments improve the results:

$ ls -lah

drwxrwxr-x  2 username group 4,0K Mar 25 12:25 .
drwxrwxr-x 16 username group 4,0K Mar 25 12:25 ..
-rwxrwxr-x  1 username group 9,1M Feb  3 17:47 jacktrip
-rwxrwxr-x  1 username group 334K Feb  3 17:47 sprawl-compositor

Create and Edit Files

A file can be created by touching it:

$ touch filename

The default editor on most systems is nano. It is a minimal terminal based tool and does not require X forwarding. To edit an existing file or create a new file, type:

$ nano filename

Terminal Only

Inside nano, you have a lot of defined keystrokes for editing, file handling and other tasks. See the nano cheat sheet for a full list: https://www.nano-editor.org/dist/latest/cheatsheet.html

GUI Based

When working with X forwarding, simple text editors with GUI features can be used. On the SPRAWL server, this includes mouspad or the minimal Python IDE idle.

Starting Applications

System wide binaries must be located in a directory that is listed in $PATH (see the final section on this page for details). They can be started by simply typing the name:

$ foo

A local binary named foo can be started with the following command:

$ ./foo

You can terminate your command with an ampersand (&) to run a process in the background. You can continue to work in the terminal, afterwards:

$ ./foo &
[1] 5459

If you start a command this way, it gives you an id of the background process in brackets and the actual process ID (PID).

You can get the process back into foreground with the fg command followed by the background process id:

$ fg 1

Check for Running Applications

At some point, users may want to know whether a process is running or which processes have been started. The command top lets you monitor the system processes with additional information on CPU and memory usage, updated with a fixed interval:

$ top

htop is a slightly polished version, using colored results:

$ htop

You can get a list of all running processes, including auxiliary ones, by typing:

$ ps aux

Usually, these are way to many results. If you want to check whether an instance of a specific program is running, you can use grep after the ps aux to filter the results:

$ ps aux | grep foo

Shell Variables

Sometimes it is convenient to store information in variables for later use. Some common variables that are used in Unix like operating systems like Linux, BSD or MacOS are for example PATH and DISPLAY.

Shell variables are usually uppercase. To get the content of a variable it is prefixed by a dollar sign. The command echo is used to print the content:

$ echo $PATH
/home/username/.local/bin:/home/username/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin
$ echo $DISPLAY
:0

Defining a variable is done with an equal sign. It happens quite often that the program that should use the variable, opens another environment. To access the variable in that sub-environment, it has to be exported before:

$ NAME=username
$ echo $NAME
username
$ bash
$ echo $NAME

$ exit
exit
$ export NAME
$ bash
$ echo $NAME
username

References

The ThroughExample

The ThroughExample is a slightly adapted version of the Simple Client. It wraps the same functionality into a C++ class, adding multi-channel capabilities.

ThroughExample *t = new ThroughExample();

jack_client_t   *client;

Standard MIDI Messages

MIDI transmits binary coded messages with a speed of $31250\ \mathrm{kbit/s}$. Timing and latency are thus not a problem when working with MIDI. However, the resolution of control values can be a limiting factor. Standard MIDI messages consist of three Bytes, namely one status Byte (first bit green) and two data bytes (first bit red). The first bit declares the Byte either a status Byte (1) or a data Byte (0).

Some of the most common messages are listed in the table below. Since one bit is used as the status/data identifier, 7 bits are left for encoding. This results in the typical MIDI resolution of \(2^7 = 128\) values for pitch, velocity or control changes.

Voice Message           Status Byte      Data Byte1          Data Byte2
-------------           -----------   -----------------   -----------------
Note off                      8x      Key number          Note Off velocity
Note on                       9x      Key number          Note on velocity
Polyphonic Key Pressure       Ax      Key number          Amount of pressure
Control Change                Bx      Controller number   Controller value
Program Change                Cx      Program number      None
Channel Pressure              Dx      Pressure value      None
Pitch Bend                    Ex      MSB                 LSB

Pitch Bend

If you are stuck with MIDI for some reason but need a higher resolution, the Pitch Bend parameter can help. Each MIDI channel has one Pitch Bend, each with two combined data Bytes, resulting in a resolution of \(128^2 = 16384\) steps.

System Exclusive

SysEx messages can be freely defined by manufacturers. They are often used for dumping or loading settings and presets, but can also be used for arbitrary control purposes. SysEx messages can have any length and are not standardized.

MIDI Note to Hertz

When working with MIDI, a conversion from MIDI pitch to Hertz is often necessary. There are two simple formulas for doing that. They both refer to the MIDI pitch of 69, wich corresponds to a frequency of 440 Hz:

Getting SC

Binaries, source code and build or installation instructions can be found at the SC GitHub site. If possible, it is recommended to build the latest version from the repository:

https://supercollider.github.io/downloads

SuperCollider comes with a large bundle of help files and code examples but first steps are usually not easy. There are a lot of very helpful additional resources, providing step by step introductions.

• The SuperCollider tutorial

• The SuperCollider Book

Code snippets in this example are taken from the accompanying repository: SC Example. You can simple copy and paste them into your editor.

SC Trinity

SuperCollider is based on a client-server paradigm. The server is running the actual audio processing, whereas clients are used to control the server processes via OSC messages. Multiple clients can connect to a running server. The dedicated ScIDE allows convenient features for live coding and project management:

$ sclang

sc3>  postln("Hello World!")

ScIDE

Working with SC in the terminal is rather inconvenient. The SuperCollider IDE (ScIDE) is the environment for live coding in sclang, allowing the control of the SuperCollider language:

When booting the ScIDE, it automatically launches sclang and is then ready to interpret. Files opened in the IDE can be executed as a whole. Moreover, single blocks, respectively single lines can be evaluated, which is especially handy in live coding, when exploring possibilities or prototyping. In addition, the IDE features tools for monitoring various server properties.

Some Language Details

(
      post('Hello ');
      postln('World!');
)

// single-letter-global variable:
x = 1.0;

// tilde-global variables:
~aValue = 1.1;

// local variable:
var foo;

(
var xValue = 1.0;

xValue.postln;

var yValue = 2.1;
)

(
~poster = {

    arg a,b;

    var y = a+b;

    y.postln;

};
)

~poster.value(1,1);

~poster = {

    |a,b|

    a.postln;

};