Application workflow
==============


This is a proposal for how we can control the sound synthesis and effects via a drawing application. Here will be discussed the ways for the users to interact with the synthesizer and the effects using the drawing application **iDraw OSC** for Apple devices.

The two possible types of interactions are:

- drawing in the drawing area;
- choosing the tool (especially the pens) and the color to draw with.

The drawing area is represented in the following figure, with the 2 parameters describing the position of the pen (X coordinate and Y coordinate) and the pressure applied to the screen. Note that if the users are drawing with the finger instead of the pen, **the pressure variable is automatically set to 1 by default**.

![Drawing area](rsc/drawing_area.jpeg)

The different tools available and the sound parameters they control are represented in the following figure.

![Different types of pens](rsc/pens.png)


In the following sections we will describe in detail all the parameters that can be controlled by the users using the drawing application. In particular, we will talk about:

- Sound synthesis: how to control the sound timbre using colors;
- Envelopes: how to control one or many envelopes using different pens;
- Effects: how to apply and control the effects by drawing



# 1) Sound synthesis

The timbre of a sound is often referred to as its color. Using this analogy, we could modify the timbre of the synth by changing the color of the pen. The color is defined by its three RGB components (red, green, blue). Let's use R, G and B to control three different waveforms, which are the three main components of our sound.


| Color         | Sound        |
| :------------ | :----------- |
| Red           | Waveform 1   |
| Green         | Waveform 2   |
| Blue          | Waveform 3   |


The output sound is the mix between Waveform 1, Waveform 2 and Waveform 3, with volumes corresponding to the intensity of the color component associated. For example using triangle, sawtooth and square as the three waveforms, we can imagine a combination like this:

> **(R, G, B) = (0.2, 0.5, 0.9) --> Output = (20% triangle, 50% sawtooth, 90% square)**



#### - Proposal 1
The choice of the waveforms is up to the users. The application lets the users choose which waveform they want to put for red, green and blue among a set of predefined waveforms. The GUI implements a way for the users to map the three RGB components to the desired waveform.

#### - Proposal 2 (Easier)
The users do not have the choice of the waveforms, they are predefined and can not be changed. In that case the waveforms should at least be very different from each other to allow the users create complex sounds.



# 2) Envelopes

Envelopes are often referred to as ADSR, standing for the four parameters Attack, Decay, Sustain, Release. While all these parameters allow the users to shape the sound as best as possible, implementing a way to control them by drawing on a pad is not an esay task, especially because the pad allows for the control of only 3 parameters: the X coordinate, the Y coordinate and the pressure. Therefore there is no way to directly map the pad parameters and the 4 parameters of the envelope.

However, we can stick to a more basic envelope, called AR (Attack, Release). This way, we can imagine the following mapping:

| Pad parameter | OSC address | Envelope parameter  | OSC range    | Envelope parameter range |
| :------------ | :---------- | :------------------ | :----------- | :----------------------- |
| X axis        | /aspectX    | Attack              | [-0.5; 0.5]  | [0; 5] seconds           |
| Y axis        | /aspectY    | Release             | [-0.5; 0.5]  | [0; 10] seconds          |

The control of the envelope can be done by selecting the appropriate pen (see below).

#### - Proposal 1
We can create 3 AR envelopes, each one controlled via a different type of pen, as follows:

| Envelope name | Envelope type | Description                                     | Pen type |
| :------------ | :----------   | :---------------------------------------------  | :------- |
| Amp. Env.     | Amplitude     | Modulates the amplitude of the sound            | Pen      |
| Filt. Env.    | Modulation    | Modulates the cutoff frequency of the LP filter | Monoline |
| Pitch. Env.   | Modulation    | Modulates the pitch of the oscillators          | Marker   |


#### - Proposal 2 (Easier)
We only create one AR amplitude envelope, controlling the amplitude of the sound, controllable with the "pen" type of pen.



# 3) Effects control

The effects can change drastically the final sound. Using the drawing surface we can apply effects and modify their parameters live. By effects we mean:

- **On SuperCollider side (applied DURING the synthesis):**
  - Low Frequency Oscillator (LFO) modulating pitch or filter;
  - Resonant filter (low-pass and high-pass)
- **On JUCE side (applied AFTER the synthesis):**
  - Distortion;
  - Reverb;
  - Chorus;
  - EQ;
  - Delay.


Let's synthesize in the following table all the possible parameters that can be modified for each one of these effects:

| Effect                | Parameters       | Description                                    |
| :-------------------- | :--------------- | :--------------------------------------------- |
| **LFO**               | LFO Rate         | Frequency of the LFO (e.g. 0.01Hz to 20Hz)
|                       | LFO Intensity    | Controls which amount of modulation we want
|                       | LFO Waveform     | Waveform for the LFO
|                       | LFO Target       | Target of the LFO
| **Filter (HP or LP)** | Cutoff frequency | Cutoff frequency
|                       | Resonance        | Q factor of the filter
| **Distortion**        | Drive            | Shape of the distortion (soft, hard...)
|                       | Dry/wet          | Controls the mix between dry and wet signal
| **Reverb**            | Room size        | How long should the reverb go
|                       | Dry/wet          | Controls the mix between dry and wet signal
| **Chorus**            | Amount           | Frequency and depth of the chorus 
|                       | Dry/wet          | Controls the mix between dry and wet signal
| **EQ**                | Low Gain         | Gain of the low shelf (< 200 Hz)
|                       | High Gain        | Gain of the high shelf (> 5000 Hz)
| **Delay**             | Amount           | Controls the amount of delay we want to apply
|                       | Dry/wet          | Controls the mix between dry and wet signal


Since the colors are used for the sound synthesis, we are left with the following controls:

- **Controls for live performance (continuous):**
  - X coordinate of the pen;
  - Y coordinate of the pen;
  - Pressure applied on the screen;
- **Controls for selection/adjusting (sent only once):**
  - Pen type;
  - Pen thickness;
  - Pen opacity;

A creative and fun interaction design is to create custom presets consisting of a combination of effects that can be modified simultaneously using the pen. Each preset can be selected using a different type of pen, thus offering 4 possibilities of presets (3 out of the 7 pen types are already used for the envelopes). The following sections describe a possible set of presets that can be good-sounding and fun for the users.



### Preset 1 - Pencil (/pencil)

This effect preset uses the LFO, the filter and the reverb as follows. The parameter ranges are given as examples but may be adjusted a posteriori.

| Pad parameter | OSC address   | Effect parameter | OSC range    | Parameter range       |
| :------------ | :------------ | :--------------- | :----------- | :-------------------- |
| X axis        | /aspectX      | Filter cutoff    | [-0.5; 0.5]  | [20; 18000] Hz
| X axis        | /aspectX      | LFO Rate         | [-0.5; 0.5]  | [0; 15] Hz
| Y axis        | /aspectY      | Filter resonance | [-0.5; 0.5]  | [0; max]
| Y axis        | /aspectY      | Reverb room size | [-0.5; 0.5]  | [min; max]
| Pen pressure  | /pressure     | LFO intensity    | [1; 8]       | [0; max]

When this preset is selected, the following parameters are set and remain the same (they are not changed in any way by the pen):

| Effect parameter | Value               |
| :--------------- | :------------------ |
| LFO Waveform     | Decreasing sawtooth |
| LFO Target       | Filter cutoff       |


### Preset 2 - Crayon (/crayon)

This effect preset uses the LFO, the filter, the reverb and the delay as follows. The parameter ranges are given as examples but may be adjusted a posteriori.

| Pad parameter | OSC parameter | Effect parameter | OSC range    | Parameter range       |
| :------------ | :------------ | :--------------- | :----------- | :-------------------- |
| X axis        | /aspectX      | LFO Rate         | [-0.5; 0.5]  | [15; 1] Hz
| X axis        | /aspectX      | LFO Intensity    | [-0.5; 0.5]  | [0; max]
| X axis        | /aspectX      | Delay amount     | [-0.5; 0.5]  | [0; max]
| Y axis        | /aspectY      | Filter cutoff    | [-0.5; 0.5]  | [20; 18000] Hz
| Y axis        | /aspectY      | Reverb dry/wet   | [-0.5; 0.5]  | [dry; dry+wet]
| Pen pressure  | /pressure     | Filter resonance | [1; 5]       | [0; max]

When this preset is selected, the following parameters are set and remain the same (they are not changed in any way by the pen):

| Effect parameter | Value               |
| :--------------- | :------------------ |
| LFO Waveform     | Square              |
| LFO Target       | Filter cutoff       |



### Preset 3 - Fountain Pen (/fountainPen)

TBD


### Preset 4 - Water Color (/waterColor)

TBD

