Andy Hulstkamp

about creating online experiences

12. September 2008

Generating Sounds. Waveforms, Timbre Loundness and Pitch.

It’s been quite a while (back in 2002, flash 5) since I did some serious work using oscillators and waveforms. I’m writing this down while trying to get back on track. The stuff described here is definitely no secret. It’s basic, but relevant to create some tunes in flash or whatever technology you might want to use. Hopefully you’ll find some of it useful.

Through the introduction of the SampleDataEvent it is now easy to write data back to the audio stream:

//const AMP_MULTIPLIER:Number = 0.15; //keep this low while testing
private function noiseWave(event:SampleDataEvent):void {   
    var sample:Number;
    for (var i:int=0; i<8192; i++ ) {
        sample = Math.random() -.5; 
        event.data.writeFloat(sample * AMP_MULTIPLIER);
        event.data.writeFloat(sample * AMP_MULTIPLIER);
    }    
}

This will simply generate some noise. Sounds rather boring, doesn’t it? Don’t neglect the noise though, it’s a great source for percussive sounds, when treated accordingly. Anyway, to get something more exciting we need to introduce pitch, timber (quality of sound) and loudness.

Tones & Pitch

To produce a sound we perceive as a tone at a certain pitch, we need to introduce a bunch of data that is repeated over time. This gives us a repeating pattern. Let‘s call a cycle of that pattern the waveform. Aha, so let’s simply write 1,-1,1,-1,1,-1… to the stream, right? Yes, we could do that, but no, you and I wouldn't hear a bleep. It would simply be out of our hearing range. To get the tones into the hearing range, the pattern needs to be repeated a certain amount of times per second. Remember the note, your music teacher in school was desperately hammering on his piano, to get everyone in tune? That was probably an A above middle C. This note has a frequency of 440 Hz, meaning the waveform (or data in our case) cycles – or oscillates - at 440 times per second. The next lower A is at 220 Hz, the next higher 880 Hz. Basically, the more cycles per sec, the higher the frequency, the higher the pitch.

Flash works at a sampling rate of 44100 Hz. It uses 44100 samples/sec to produce sound. If we want to generate an A (440 Hz) in flash, we need to repeatedly write a bunch of 44100/440 ≈ 100 samples to the audio stream. The 100 samples are actually the wavelength represented in samples.

//const AMP_MULTIPLIER:Number = 0.15;
//const BASE_FREQ:int = 440;
//const SAMPLING_RATE:int = 44100;
//const TWO_PI:Number = 2*Math.PI;
//const TWO_PI_OVER_SR:Number = TWO_PI/SAMPLING_RATE;

private function sineWave1(event:SampleDataEvent):void {
    var sample:Number
    for (var i:int=0; i<8192; i++) {
        sample = Math.sin((i+event.position) * TWO_PI_OVER_SR * BASE_FREQ);
        event.data.writeFloat(sample * AMP_MULTIPLIER);
        event.data.writeFloat(sample * AMP_MULTIPLIER);
    }
}

This will generate a pure sine A440Hz. By layering multiple sines of different frequencies a vast number of waveforms can be created. A square wave could be generated using multiple sines, but that would put heavy loads on the CPU. Luckily we can cheat:

//the square wave implemented via sine, duty cycle for a square is 1:2
private function squareWave1(event:SampleDataEvent):void {
    //lets be nice to our equipment and keep the amplitude low
    var amp:Number = 0.075;
    var sample:Number;
    for (var i:int=0; i<8192; i++) {
        sample = Math.sin((i + event.position) * TWO_PI_OVER_SR * BASE_FREQ) > 0 ? amp : -amp;
        event.data.writeFloat(sample);
        event.data.writeFloat(sample);
    }
}

This gives us a nice square wave, but there’s quite an expensive Math.sin() in it. Another approach might be better regarding performance:

//keeps track of current phase
var phase:Number;

private function squareWave2(event:SampleDataEvent):void {
    var amp:Number = 0.075;
    var sample:Number;
    for (var i:int; i < 8192; i++) {
        sample = phase < Math.PI ? amp : -amp;
        phase = phase + (TWO_PI_OVER_SR * BASE_FREQ);
        phase = phase > TWO_PI ? phase-TWO_PI : phase;
        event.data.writeFloat(sample);
        event.data.writeFloat(sample);
    }
}

The square wave has the same pitch as the sine wave above but sounds different (different quality of sound or timbre). Why? Overtones.

Timbre (quality of sound)

The perceived timbre of a sound is determined by its spectrum and loudness over time. The spectrum is basically the sum of different frequencies in a sound. Based on the fundamental frequency (in our example 440 Hz) there could be other frequencies (overtones) on top of that. These overtones can either be harmonic, when they are multiple octaves above the fundamental frequency or in-harmonic, when somewhere in-between. We don’t need to deal with overtones, let’s just state that the waveform and timbre of a sound are coupled. HOW a waveform sounds is mainly a question of perception and association. The other factor that defines the timbre is the loudness of a sound over time.

Loudness

Loudness of a sound over time is another factor that defines the timbre. If we change

-this:
event.data.writeFloat(sample);
event.data.writeFloat(sample);
-to that:
event.data.writeFloat(sample * (8192-c)/8192);
event.data.writeFloat(sample * (8192-c)/8192);

we get a sound with a strong attack and a linear decay. However, the sounds we got so far are a little static. To get more interesting results, we need to introduce modulation. Modulation can affect all kinds of parameters (like pitch, loudness, envelope, shifts in overtones, anything) to slightly or drastically change the waveform over time. Here’s an example that modulates the pulse width and the amplitude of a pulse wave:

//modulated pulse, one LFO modulates pulse-width, another one the amplitude
private function pulseWaveMod1(event:SampleDataEvent):void {
         // get those out
        var amp:Number = 0.075;
    var pwr:Number = Math.PI/1.05;
    var dpw:Number;
    var am:Number;
    var pos:Number;
    var sample:Number;
    for (var i:int=0; i<8192; i++) {
        pos = i + event.position;
        dpw = Math.sin (pos/0x4800) * pwr; //LFO -> PW
        sample = phase < Math.PI - dpw ? amp : -amp;
        phase = phase + (TWO_PI_OVER_SR * BASE_FREQ);
        phase = phase > TWO_PI ? phase-TWO_PI : phase;
        am = Math.sin (pos/0x1000); //LFO -> AM
            event.data.writeFloat(sample * am);
        event.data.writeFloat(sample * am);
    }
}

By modulating different parameters the sound gets more interesting. Now, nobody stops you from modulating the modulators, giving more drastic results.

Here’s a little test of basic waveforms up to rather heavily modulated:

Flash Waveform And Sound Tester

Turn down volume first. Needs the latest Flash-Player 10 (rc 091508).

Here’s the source. UPDATE: Adobe changed the sound api. Use SampleDataEvent.SAMPLE_DATA instead of Event.SAMPLE_DATA for the listener.

Yeah, sounds still thin, though

You’re right. To get those fat bastard sounds we need more voices, effects (the holy reverb), filters, compressors etc. but that’s a whole different story. In the end, at the basis of it all are the waveforms.

further reading

Custom Spark CheckBox Component in Flex 4 (Gumbo)

Post abput how to customize the Flex Spark Checkbox.

Custom Component using Flex SpriteVisualElement

A much more lightweight approach to create a simple custom Component in Flex Spark. Does not extend from the heavier SkinnableComponent but from SpriteVisualElement.