|
|
@@ -0,0 +1,135 @@
|
|
|
+Application workflow
|
|
|
+==============
|
|
|
+
|
|
|
+
|
|
|
+This is a proposal for how we can control the sound synthesis and effects via a drawing application.
|
|
|
+
|
|
|
+
|
|
|
+# 1) Sound synthesis
|
|
|
+
|
|
|
+The timbre of a sound is often referred to as its color. Using this analogy, we could modify the timbre of the synth by changing the color of the pen. The color is defined by its three RGB components (red, green, blue). Let's use R, G and B to control three different waveforms, which are the three main components of our sound.
|
|
|
+
|
|
|
+
|
|
|
+| Color | Sound |
|
|
|
+| :------------ | :----------- |
|
|
|
+| Red | Waveform 1 |
|
|
|
+| Green | Waveform 2 |
|
|
|
+| Blue | Waveform 3 |
|
|
|
+
|
|
|
+
|
|
|
+The output sound is the mix between Waveform 1, Waveform 2 and Waveform 3, with volumes corresponding to the intensity of the color component associated. For example using triangle, sawtooth and square as the three waveforms, we can imagine a combination like this:
|
|
|
+
|
|
|
+> **(R, G, B) = (0.2, 0.5, 0.9) --> Output = (20% triangle, 50% sawtooth, 90% square)**
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+#### - Proposal 1
|
|
|
+The choice of the waveforms is up to the users. The application lets the users choose which waveform they want to put for red, green and blue among a set of predefined waveforms. The GUI implements a way for the users to map the three RGB components to the desired waveform.
|
|
|
+
|
|
|
+#### - Proposal 2 (Easier)
|
|
|
+The users do not have the choice of the waveforms, they are predefined and can not be changed. In that case the waveforms should at least be very different from each other to allow the users create complex sounds.
|
|
|
+
|
|
|
+
|
|
|
+# 2) Effects control
|
|
|
+
|
|
|
+The effects can change drastically the final sound. Using the drawing surface we can apply effects and modify their parameters live. By effects we mean:
|
|
|
+
|
|
|
+- **On SuperCollider side (applied DURING the synthesis):**
|
|
|
+ - Low Frequency Oscillator (LFO) modulating pitch or filter;
|
|
|
+ - Resonant filter (low-pass and high-pass)
|
|
|
+- **On JUCE side (applied AFTER the synthesis):**
|
|
|
+ - Distortion;
|
|
|
+ - Reverb;
|
|
|
+ - Chorus;
|
|
|
+ - EQ;
|
|
|
+ - Delay.
|
|
|
+
|
|
|
+
|
|
|
+Let's synthesize in the following table all the possible parameters that can be modified for each one of these effects:
|
|
|
+
|
|
|
+| Effect | Parameters | Description |
|
|
|
+| :--------------- | :--------------- | :--------------------------------------------- |
|
|
|
+| LFO | LFO Rate | Frequency of the LFO (e.g. 0.01Hz to 20Hz)
|
|
|
+| | LFO Intensity | Controls which amount of modulation we want
|
|
|
+| | LFO Waveform | Waveform for the LFO
|
|
|
+| | LFO Target | Target of the LFO (pitch or filter)
|
|
|
+| Filter (HP or LP) | Cutoff frequency | Cutoff frequency
|
|
|
+| | Resonance | Q factor of the filter
|
|
|
+| Distortion | Input level | Level of the input signal (before distortion)
|
|
|
+| | Drive | Shape of the distortion (soft, hard...)
|
|
|
+| | Dry/wet | Controls the ratio between input signal and signal with distortion
|
|
|
+| Reverb | Room size | How long should the reverb go
|
|
|
+| | Dry/wet | Controls the amount of reverb we want to apply
|
|
|
+| Chorus | Amount | Frequency and depth of the chorus
|
|
|
+| | Dry/wet | Controls the amount of chorus we want to apply
|
|
|
+| EQ | Type | Type of filter (shelf, peak/notch, ...)
|
|
|
+| | Frequency | Central frequency of the filter
|
|
|
+| | Gain | Gain to apply
|
|
|
+| Delay | Time | Time between each repetition of the sound
|
|
|
+| | Intensity | Intensity of each repetition of the sound
|
|
|
+
|
|
|
+
|
|
|
+Since the colors are used for the sound synthesis, we are left with the following controls:
|
|
|
+
|
|
|
+- **Controls for live performance (continuous):**
|
|
|
+ - X coordinate of the pen;
|
|
|
+ - Y coordinate of the pen;
|
|
|
+ - Pressure applied on the screen;
|
|
|
+- **Controls for selection/adjusting (sent only once):**
|
|
|
+ - Pen type;
|
|
|
+ - Pen thickness;
|
|
|
+ - Pen opacity;
|
|
|
+
|
|
|
+Instead of applying each one of these effects individually, a more creative and fun interaction design is to create custom presets consisting of a combination of effects that can be modified simultaneously using the pen. Each preset can be selected using a different type of pen, thus offering 7 possibilities of presets (one per type of pen). The following sections describe a possible set of presets that can be good-sounding and fun for the users.
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+### Preset 1 - Regular pen (/pen)
|
|
|
+
|
|
|
+This effect preset uses the LFO, the filter and the reverb as follows:
|
|
|
+
|
|
|
+| Pad parameter | Effect parameter |
|
|
|
+| :------------ | :--------------- |
|
|
|
+| X axis | Filter cutoff |
|
|
|
+| X axis | LFO Rate |
|
|
|
+| Y axis | Filter resonance |
|
|
|
+| Y axis | Reverb depth |
|
|
|
+| Pen pressure | LFO intensity |
|
|
|
+
|
|
|
+When this preset is selected, the following parameters are set and remain the same (they are not changed in any way by the pen):
|
|
|
+
|
|
|
+| Effect parameter | Value
|
|
|
+| :--------------- | :------------------ |
|
|
|
+| LFO Waveform | Decreasing sawtooth |
|
|
|
+| LFO Target | Filter cutoff |
|
|
|
+
|
|
|
+
|
|
|
+### Preset 2 - Pencil (/pencil)
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+### Preset 3 - Marker (/marker)
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+### Preset 4 - Monoline (/monoline)
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+### Preset 5 - Crayon (/crayon)
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+### Preset 6 - Fountain Pen (/fountainPen)
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+### Preset 7 - Water Color (/waterColor)
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+
|