RingBuffer (Posts about _sound_synthesis:fm-synthesis)

FM Synthesis: Python Example

Henrik von Coler — Wed, 02 Apr 2025 10:11:12 GMT

This example shows a simulation of a DX7-inspired 2-operator FM synthesizer. The implementation is based on three classes:

Envelope Class
Operator Class
Synthesizer Class

This class structure allows the programming of the synth with different alorithms, that define the signal flow between the operators.

Envelope Class¶

The DX7Envelope class emulates the envelope generator of the DX7 synthesizer. It includes four levels (Attack, Decay, Sustain, and Release) and four rates - the corresponding durations. The generate() function creates the envelope over a specified duration.

In [122]:

import numpy as np
import scipy as sp
import matplotlib.pyplot as plt

In [123]:

class DX7Envelope:
    
    """DX7-Style Envelope with 4 Levels and 4 Times"""
    def __init__(self, levels=[1.0, 0.8, 0.5, 0.0], times=[0.1, 0.3, 0.5, 0.7], fs=48000):
        self.fs = fs
        self.set_envelope(levels, times)

    def set_envelope(self, levels, times):
        """Set DX7 envelope levels and times dynamically"""
        if len(levels) != 4 or len(times) != 4:
            raise ValueError("DX7 envelopes must have exactly 4 levels and 4 times.")
        self.levels = levels
        self.times = times

    def generate(self, duration):
        """Generate the envelope over the given duration"""
        total_samples = int(self.fs * duration)
        env = np.zeros(total_samples)

        # Convert times to sample counts
        time_samples = [int(t * self.fs) for t in self.times]

        # Build the envelope shape
        start_idx = 0
        
        # Attack phase (Starts at 0 and ramps to first level)
        end_idx = start_idx + time_samples[0]
        env[start_idx:end_idx] = np.linspace(0, self.levels[0], end_idx - start_idx)
        start_idx = end_idx

        # Decay phase (from level 1 to level 2)
        end_idx = start_idx + time_samples[1]
        env[start_idx:end_idx] = np.linspace(self.levels[0], self.levels[1], end_idx - start_idx)
        start_idx = end_idx

        # Sustain phase (constant level)
        end_idx = start_idx + time_samples[2]
        env[start_idx:end_idx] = self.levels[1]  # Sustain stays constant
        start_idx = end_idx

        # Release phase (from sustain level to 0)
        release_idx = sum(time_samples[0:-1])
        env[release_idx:] = np.linspace(self.levels[1], self.levels[3], total_samples - release_idx)

        return env

Example Envelope¶

The following plot shows an example envelope with a slight attack, medium decay, a strong sustain and a long release segment:

In [124]:

# Create an ADSR envelope
levels = [1.0, 0.8, 0.5, 0.0]  # Levels: Attack, Decay, Sustain, Release
times  = [0.1, 0.3, 0.5, 0.7]   # Durations: Attack, Decay, Sustain, Release

# Initialize DX7 envelope
adsr = DX7Envelope(levels=levels, times=times)
# Generate the envelope (2 seconds duration)
duration = 2.0
env = adsr.generate(duration)




# Plot the ADSR envelope with markers and level lines
t = np.linspace(0, duration, len(env))

plt.figure(figsize=(8, 3))
plt.plot(t, env, label="ADSR Envelope", color='k')
plt.title("ADSR Envelope with Level Lines and Labels")
plt.xlabel("Time (seconds)")
plt.ylabel("ADSR")

# Mark the phases with labels
phase_labels = ["RATE 1", "RATE 2", "RATE 3", "RATE 4"]
time_labels = [0, times[0], times[0] + times[1], times[0] + times[1] + times[2], duration]
level_labels = [levels[0], levels[1], levels[1], levels[3]]

# Add markers for each phase change
for i, label in enumerate(phase_labels):
    plt.axvline(x=time_labels[i+1], color='gray', linestyle='--', label=label)
    plt.text(time_labels[i+1], -0.5, f'{label}\n({time_labels[i]:.2f}s)', horizontalalignment='center')

# Plot the level lines (horizontal lines at each level)
for i, level in enumerate(levels):
    plt.axhline(y=level, color='gray', linestyle=':', linewidth=1)  # Horizontal line at the level
    plt.text(-0.25, level, "LEVEL "+str(i+1), verticalalignment='center', horizontalalignment='right')

#plt.legend()
plt.tight_layout()
plt.xlim([0,2])
plt.ylim([0,1.1])
plt.show()

Operator Class¶

The FMOperator class simulates a single FM operator, similar to the ones used in the DX7. It generates a sine wave signal that can optionally be modulated by another operator. The generate() function creates the modulated signal based on the operator's frequency, modulation index, and the envelope.

In [125]:

class FMOperator:
    
    def __init__(self, frequency, fs=48000, envelope=None, mod_index=1.0):
        self.frequency = frequency
        self.fs = fs
        self.mod_index = mod_index
        self.envelope = envelope if envelope else DX7Envelope()

    def generate(self, duration, modulator=None):
        t = np.linspace(0, duration, int(self.fs * duration), endpoint=False)
        env = self.envelope.generate(duration)

        mod_signal = modulator.generate(duration) * self.mod_index if modulator else np.zeros_like(t)
        signal = np.sin(2 * np.pi * self.frequency * t + 2 * np.pi * mod_signal)

        return signal * env

Synth Class¶

The DX7Synth class combines two FM operators, each with its own frequency, envelope, and modulation index. The class allows for the creation of a synthesis patch and generates the final output by combining the signals from the operators according to a modulation algorithm.

In [126]:

class DX7Synth:
    
    def __init__(self, fs=48000):
        self.fs = fs
        self.operators = [FMOperator(frequency=440, fs=fs) for _ in range(2)]  # 2 operators
        self.current_algorithm = [[0, 1]] 
        self.current_note = None

    def set_patch(self, freqs, envelopes, mod_indices):        
        for i in range(2):  # Adjust to 2 operators
            self.operators[i].frequency = freqs[i]
            self.operators[i].envelope = envelopes[i]
            self.operators[i].mod_index = mod_indices[i]

    def generate(self, duration):
        op_outputs = [op.generate(duration, modulator=self.operators[0]) for op in self.operators]

        # Combine outputs based on the algorithm
        final_output = np.zeros_like(op_outputs[0])

        # Apply the modulation algorithm
        for mod, carrier in self.current_algorithm:
            final_output += op_outputs[carrier]  # Carrier adds modulated signal to final output

        return final_output

Programming a Metallic Sound¶

We will use the two operatores to create a metallic sound, with the algorithm shown in the following flowchart.

OP1:
- Has a fixed frequency $f_1$.
- Is not being modulated ($I_1$ has no effect).
- Its amplitude is controlled by ADSR$_1$
OP2:
- Has a central frequency $f_2$.
- Is modulated by the output of OP1.
- The modulation index $I_2$ determindes the strenght of the modulation.
- The ADSR$_1$ determines the amplitude envelope of the final output.

Both envelopes are triggered at the same instance - denoted as $T$, for trigger.

In [56]:

freqs = [750, 955]       # operators are not in harmonic ratio to create metallic sound
mod_indices = [0.0, 1.0]  
envelopes = [
    DX7Envelope([1.0, 0.2, 0.1, 0.05], [0.0, 0.4, 0.1, 0.25], fs=48000),   
    DX7Envelope([1.0, 0.2, 0.3, 0], [0.0, 0.2, 0.11, 0.123], fs=48000), 
]

synth = DX7Synth()

# Set the patch
synth.set_patch(freqs, envelopes, mod_indices)

duration = 1.5
sound    = synth.generate(duration)

Plotting Spectrograms and Envelopes¶

We can visualize the different stages by plotting the spectrograms of the operators and the final output, next to the operators' envelopes. While the operators' spectrograms show only one horizontal line for their frequency, the final spectrogram shows decazing overtones, caused by the trajectory of ADSR$_1$.

In [121]:

# Time axis for plotting
t = np.linspace(0, duration, int(synth.fs * duration), endpoint=False)

# Create a plot for each operator's spectrogram and envelope
fig, axes = plt.subplots(2, 2, figsize=(12, 6))

# Plot the spectrogram and envelope for each operator
for i in range(2):
    # Generate the operator's signal
    op_signal = synth.operators[i].generate(duration)
    
    # Plot the spectrogram of the operator
    axes[i, 0].specgram(op_signal, NFFT=2*1024, Fs=synth.fs, noverlap=512)
    axes[i, 0].set_title(f"Operator {i+1} Spectrogram")
    axes[i, 0].set_xlabel("Time (seconds)")
    axes[i, 0].set_ylabel("Frequency (Hz)")
    axes[i, 0].set_ylim([0,5000])
    
    # Plot the envelope of the operator
    envelope = envelopes[i].generate(duration)
    axes[i, 1].plot(t, envelope)
    axes[i, 1].set_title(f"Operator {i+1} Envelope")
    axes[i, 1].set_xlabel("Time (seconds)")
    axes[i, 1].set_ylabel("Amplitude")
    

plt.tight_layout()
plt.show()

fig, axes = plt.subplots(1, 1, figsize=(6, 3))

plt.specgram(sound, NFFT=2*1024, Fs=synth.fs, noverlap=512)
# Plot the final output spectrogram
plt.title("Output Spectrogram")
plt.xlabel("Time (seconds)")
plt.ylabel("Frequency (Hz)")
plt.ylim([0,20000])

plt.show()

Resulting Sound¶

The resulting sound has the metallic sound, that is characteristic for FM synthesis:

In [55]:

from IPython.display import Audio
Audio(sound, rate=synth.fs)

Out[55]:

Your browser does not support the audio element.

FM Synthesis: DX7

Henrik von Coler — Mon, 11 May 2020 16:23:32 GMT

FM synthesis was not only an outstanding method for experimental music but landed a major commercial success. Although there are many more popular and valuable synthesizers from the 80s, no other device shaped the sound of pop music in that era like the DX7 did. It was not the first ever, but the first affordable FM-capable synth and can generate a wide variety of sounds -- bass, leads, pads, strings, ... -- with extensive (but complicated) editing opportunities. It was also the breakthrough of digital sound synthesis, using the full potential with MIDI.

[Fig.1]

Yamaha DX7.

Specs

released in 1983
16 Voices Polyphony
6 sine wave 'operators' per voice
velocity sensitive
aftertouch
LFO
MIDI

The DX7 in 80s Pop

Tina Turner - What's Love Got To Do With It

1984

blues harp preset

starting 2:00

Laura Branigan - Self Control

1984

the bells

Harold Faltenmeyer - Axel F

1986

marimbas

starting 1:40

Kenny Loggins - Danger Zone

1986

FM bass

A Comprehensive List

Find a comprenesive list of famous examples, here:

http://bobbyblues.recup.ch/yamaha_dx7/dx7_examples.html

Programming the DX7

The DX7 can be fully programmed using membrane buttons. Alternatively, Sysex messages can be used to work with external programmers, like a laptop, over MIDI. For users new to FM synthesis, it may be confusing not to find any filters. Timbre is solely controlled using the FM parameters, such as operator freuqncy ratios and modulation indices.

Algorithms

The configuration of the six operators, respectively how they are connected, is called algorithm in the Yamaha terminology. In contrast do some of its successors, the DX7 does not allow the free editing of the operator connections but provides a set of 32 pre-defined algorithms, shown in [Fig.2].

[Fig.2]

Yamaha DX7 manual: algorithm selection.

Envelopes

For generating sounds with evolving timbres, each operator's amplitude can be modulated with an individual ADHSR envelope, shown in [Fig.3]. Depending on the algorithm, this directly influences the modulation index and thus the overtone structure.

[Fig.3]

Yamaha DX7 manual: envelope editing.

Velocity

The level of each operator, and therefor modulation indices, can be programmed to depend on velocity. This allows the timbre to depend on the velocity, as in most physical instruments, which is crucial for expressive performances.

FM Synthesis: Interactive Example

Henrik von Coler — Mon, 11 May 2020 12:06:11 GMT

The following example is a minimal FM synthesis with two operators - one modulator and one carrier:

Carrier (Hz):

Modulator (Hz):

Modulation Depth (Hz):

Gain:

Time Domain:

Frequency Domain:

FM Synthesis: Formula & Spectrum

Henrik von Coler — Mon, 11 May 2020 12:06:11 GMT

FM Equation¶

Frequency modulation with two sinusoidal oscillators can be written as follows:

$$ x(t) = A \sin(2 \pi f_c t + I \sin(2 \pi f_m t) ) $$

The modulation index $I$ determins the strength of the modulation effect, relative to the modulator frequency. It can be expressed as:

$$I = \frac{\Delta f}{\Delta f_m}$$

Spectrum of Frequency Modulation¶

Compared to amplitude modulation techniques, FM generates more - as in infinite - spectral components, which can be illustrated when calculating the Fourier transform of the FM formula:

Based on the FM formula, the spectrum can be calculated using trigonometric identities:

$$ x(t) = \cos(\omega_\alpha t + I \sin(\omega_\beta t) ) $$

$\textbf{with:} \cos(a+b) = \cos(a) \cos(b) - \sin(a) \sin(b)$

$$ x(t) = \cos(\omega_\alpha t) \cos(I \sin(\omega_\beta t)) - \sin(\omega_\alpha t) \sin(I \sin(\omega_\beta t)) $$

$\textbf{With:} \cos(a) \cos(b) = \frac{1}{2} \left( \cos(a-b) + \cos(a+b) \right)$

$\textbf{And:} \sin(a) \sin(b) = \frac{1}{2} \left( \cos(a-b) - \cos(a+b) \right)$

$$ \begin{eqnarray} &=& \frac{1}{2} ( \sin(\omega_\alpha t + I \sin(\omega_\alpha t)) \\ &&+ \sin(\omega_\alpha t - I \sin(\omega_\alpha t))\\ &&+ \sin(\omega_\alpha t + I \sin(\omega_\alpha t)) \\ &&+ \sin( I \sin(\omega_\alpha t) - \omega_\alpha t) ) \end{eqnarray} $$

Bessel functions¶

The Fourier transform of the equation above can be expressed through a series of delta impulses, with amplitudes being derived from Bessel functions $J_n$:

$$X(f) = \sum_{n = -\infty}^{\infty} J_n^2(\beta) \delta(f - f_c - n f_m)$$

Since this series is infinite, the spectrum of FM signals has an infinite number of sidebands which are increased in energy for high modulation indices.

Bessel Functions of first Kind¶

Harmonic vs Inharmonic¶

Depending on the ratio between modulator and carrier, the FM spectrum has different properties. In the formula for the FM sidebands is is obvious that for integer ratios between carrier to modulator, all sidebands are integer multiples of the fundamental frequency.

FM Synthesis: History & Backgroud

Henrik von Coler — Mon, 11 May 2020 14:50:50 GMT

A Brief History

John Chowning brought a copy of the MUSIC IV software from Bell Labs to Stanford, where he founded the CCRMA, and started experiments in sound synthesis. Although frequency modulation was already a method used in analog sound synthesis, it was Chowning who developed the concept of frequency modulation (FM) synthesis with digital means in the late 1960s.

The concept of frequency modulation, already used for transmitting radio signals, was transferred to the audible domain by John Chowning, since he saw the potential to create complex (as in rich) timbres with a few operations (Chowning, 1973). For one sinusoid modulating the frequency of a second, frequency modulation can be written as:

$$y(t) = \sin(2 \pi f_c + I_m \sin(2 \pi f_m t))$$

$f_c$ denotes the so carrier frequency.

$f_m$ the modulation frequency.

$I_m$ is the modulation index.

Signal Flow in MUSIC IV

[Fig.1] shows a flow chart for this operation in the style of MUSIC IV.

[Fig.1]

Flow chart for FM with two operators (Chowning, 1973).

Sound Design with Envelopes

In many musical applications, the use of dynamic spectra is desirable. The parameters of the above shown FM algorithm are therefor controlled with temporal envelopes, as shown in [Fig.2]. Especially the change of the modulation index over time is important, since it results in percussive sound qualities. In musical applications, multiple carriers and modulators, referred to as operators, are connected in different configurations, for generating richer timbres.

[Fig.2]

Flow chart for dynamic FM with two operators (Chowning, 1973).

FM synthesis is considered an abstract algorithm. It does not come with a related analysis approach to generate desired sounds but they need to be programmed or designed. However, there are attempts towards an automatic parametrization of FM synthesizers (Horner, 2003).

Early Experimental Compositions

John Chowning, composer by profession, combined the novel FM synthesis approach with digital spatialization techniques to create quadraphonic pieces of electronic music on a completely new level. In Turenas, completed in 1972, artificial doppler shifts and direct-to-reverberation techniques are used to intensify the perceived motion and distance of panned sounds in the loudspeaker setup.

The sounds used in this piece are only generated by means of FM, resulting in a characteristic quality like the synthetic bell-like sounds beginning at 1:30 or the re-occuring short precussive events. Another prominent example is Stria (1977):

References

2011

John Chowning. Turenas: the realization of a dream. Proc. of the 17es Journées d’Informatique Musicale, Saint-Etienne, France, 2011.
[details] [BibTeX▼]

@article{chowning2011turenas,
    author = "Chowning, John",
    journal = "Proc. of the 17es Journées d’Informatique Musicale, Saint-Etienne, France",
    title = "{Turenas: the realization of a dream}",
    year = "2011"
}

2003

Andrew Horner. Auto-programmable FM and wavetable synthesizers. Contemporary Music Review, 22(3):21–29, 2003.
[details] [BibTeX▼]

@article{horner2003auto,
    author = "Horner, Andrew",
    journal = "Contemporary Music Review",
    number = "3",
    pages = "21–29",
    publisher = "Taylor \\& Francis",
    title = "{Auto-programmable FM and wavetable synthesizers}",
    volume = "22",
    year = "2003"
}

1973

John M Chowning. The synthesis of complex audio spectra by means of frequency modulation. Journal of the audio engineering society, 21(7):526–534, 1973.
[details] [BibTeX▼]

@article{chowning1973thesynthesis,
    author = "Chowning, John M",
    journal = "Journal of the audio engineering society",
    number = "7",
    pages = "526–534",
    publisher = "Audio Engineering Society",
    title = "{The synthesis of complex audio spectra by means of frequency modulation}",
    volume = "21",
    year = "1973"
}