From the partial tracking we have three matrices that describe the evolution of a sound over time:
-
The frequency matrix: $ \mathbf F_{n,i} $
-
The amplitude matrix: $ \mathbf A_{n,i} $
-
The phase matrix: $ \mathbf P_{n,i} $
Each of these matrices has a the time along one dimension, with the frame index $n$, and the partials along the other dimension, with the partial index $i$.
Oscillator Bank¶
The simplest way to re-synthesize a sound after the analysis process is in the time-domain - with an oscillator bank - which is basically an array of oscillators. In this case we are ignoring the original phase from the analysis.
The figure below shows this for an example with three partials. Each oscillator is driven with its frequency trajectory $f_i(t)$ of a single partial. Each partial's gain is then controlled by the relevant partial amplitude trajectory $a_i(t)$. The resulting components are then added to create the complete harmonic signal $y(t)$.
Trajectory Interpolation¶
Our matrices $F_{n,i}$ and $A_{n,i}$ have values only for each STFT frame of the analysis process. They are $L_{HOP}$ samples apart. That means we need to interpolate all partial control trajectories ($f_i(t)$ and $a_i(t)$) between these support.
The following plot shows how a partial amplitude trajectory $a_i(t)$ is created through interpolation from matrix values, with an analysis hop-size of $0.1$ seconds:
