Tag Archives: structuredaudio

Exit Structured Audio, Enter Home Grown

In a previous blog post, I had explored using MPEG4 Structured Audio to produce the sounds that my robots will make in future animation.  A few weeks later, I find that the learning curve for the tools is a bit too steep for my liking… so I decided to start from scratch… and write my own.

Yeah, I understand that people of far greater sound knowledge and experience put a LOT of effort into Structured Audio, but that doesn’t help me now.  Maybe this tool that I’m building (which doesn’t quite have a name yet) will iterate toward the stuff I’m giving up… but I’m sure I’ll learn quite a bit about audio programming along the way.

As it turns out, most of the tool chain that I had previously built still works! I’m just replacing sfront with something that I’ve written… and if you’re curious what a program looks like which generates .wav files? here it is!

#!/usr/bin/perl

use strict;

my $filename = shift @ARGV;
die "You must specify an output file." unless $filename;

my %aliases = (
   C4 => 261.626,
   D4 => 293.665,
   E4 => 329.628,
   F4 => 349.228,
   G4 => 391.995,
   A4 => 440.000,
   B4 => 493.883,
   C5 => 523.251
);

my @song = qw(B4 A4 G4 A4 B4 B4 B4 A4 A4 A4 B4
              D4 D4 B4 A4 G4 A4 B4 B4 B4 B4 A4 A4 B4 A4 G4);
my $duration = (scalar @song) * 0.5 + 0.5;
my $volume = 8000;

my @notes;
push @notes, $aliases{$_} for @song;

my $channels = 1;
my $samplerate = 44100;
my $bitspersample = 16;
my $byterate = $channels * $samplerate * $bitspersample / 8;
my $blockalign = $channels * $bitspersample / 8;
my $samples = $samplerate * $duration;
my $dataChunkSize = $samples * $bitspersample / 8;
my $chunkSize = 36 + $dataChunkSize;

my @values;
for (my $i = 0; $i < $samples; $i++) {
   my $note = int($i / $samplerate * 2);
   my $freq = $notes[$note];
   my $x = $i * $freq / $samplerate * 3.14159 * 2;
   push @values, int($volume * sin($x));
}

open(FILE, "> $filename") || die "could not open $filename";
print FILE pack('NVN', 0x52494646, $chunkSize, 0x57415645);
print FILE pack('NVvvVVvv', 0x666d7420, 16, 1, $channels, $samplerate,
     $byterate, $blockalign, $bitspersample);
print FILE pack('NV', 0x64617461, $dataChunkSize);
for my $sample (@values) {
    print FILE pack('v', $sample);
}

close(FILE);

Yup. That’s the whole thing.  No libraries. No weird complexity.

I decided to write the thing in perl for the practice.  I use an awful lot of perl at work, so as this project grows, it’ll force me to learn the proper perl patterns.

If you’re curious what that program produces? Here’s the converted MP3:

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Structured Audio C Major Scale

I’m continuing to experiment with MP4-SA.  This time around I have a bit more refined infrastructure built.  The major trouble that I had with my previous audio file, It needed to be refined from the raw .wav generated by sfront.

My end goal was to have a process by which I could write instruments and score files and in a single command produce a properly ID3 annotated mp3 file that I could then use to play via the WordPress Audio Player.

Using macports, I was able to quickly install Blade’s MP3 Encoder, and id3tool.

Once all of these packages were installed, I wrote a simple template bash script that takes all of these disparate pieces and forms voltron.

The result is the following, very modest, audio file rendering a simple C Major scale…

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Yeah it’s still rough… I’m not entirely sure what the popping sounds are after each note.  It probably has to do with the signal being cut off abruptly… That’s something I’ll have to work out in the instrument file itself…

Ok. Well, what does a C Scale look like in MP4-SA?

0.00  @INSTRUMENT@ 0.40 261.626 0.8
0.50  @INSTRUMENT@ 0.40 293.665 0.8
1.00  @INSTRUMENT@ 0.40 329.628 0.8
1.50  @INSTRUMENT@ 0.40 349.228 0.8
2.00  @INSTRUMENT@ 0.40 391.995 0.8
2.50  @INSTRUMENT@ 0.40 440.000 0.8
3.00  @INSTRUMENT@ 0.40 493.883 0.8
3.50  @INSTRUMENT@ 0.40 523.251 0.8
4.50 end

The strange @INSTRUMENT@ markup is part of my tool chaining.  Prior to running the sfront program on this file, my shell script replaces all occurrences of @INSTRUMENT@ with the actual instrument being played… in this case it was a sine wave.  The first number after the instrument turns out to be duration of tone.  In my last post I had mistaken this field to be stop time.  The 2nd value is frequency.  This has also changed from my first attempt… instead of a fairly arbitrary decimal value, this is an actual measure of frequency in Hertz.  Finally we have amplitude which pretty much changed the same…

To facilitate the changes in what the parameters mean, I also changed the basic sine instrument that I wrote.  Here is the new, improved sine

instr sine(f, a) {
   asig x, y, init, arate;
   if (init == 0) {
      init = 1;
      x = 0.0;
      arate= f * 3.14159 * 2 / 32000;
   }
   x = x + arate;
   y = sin(x);
   output(y * a);
}

You can see here, that I’m actually using a for real sine function now. The arate variable computes the x value of sine based on the sample rate of the composition, in this case 32000.  In reality, this should be a global variable of some kind but I could not seem to get that to work.

One thing I will probably do in the future is change the frequency numbers of each note to be aliases, much like the @INSTRUMENT@.  This will make it easier to score more complex songs.  I should also do that for note duration.

Messing with Structured Audio

It’s time to seriously consider what Seña will sound like. To accomplish that, I’ve started messing with structured audio.  What does that mean?

It seems that part of the MP4 standard includes a section involving programmatically generated sound.  Basically, making synthetic sounds with programming.  It’s visually pretty drab, but if you’re into coding like I am… it’s pretty cool.

The audio programming language takes 2 major components, a SAOL file, and a SASL file.  The SAOL file is where you define your “instruments” and the SASL is where you sequence your instruments playing notes…

I made a test SASL and SAOL based on the example scripts they provided.

instr sine_tone(f, a) {
    asig x, y, init;

    if (init == 0) {
        init = 1;
        x = 0.5;
    }

    x = x - f * y;
    y = y + f * x;

    output(y * a);
}

Here I added two parameters, f and a. This allowed me to control the frequency and amplitude of the approximated sine wave created by the x= and y= expressions.

To actually play the instrument, I made a SASL file:

0.25 sine_tone 0.50 0.196307 0.10
0.50 sine_tone 0.75 0.296307 0.10
0.75 sine_tone 1.00 0.396307 0.10
1.25 sine_tone 1.50 0.196307 0.20
1.50 sine_tone 1.75 0.296307 0.20
1.75 sine_tone 2.00 0.396307 0.20
2.25 sine_tone 2.50 0.196307 0.35
2.50 sine_tone 2.75 0.296307 0.35
2.75 sine_tone 3.00 0.396307 0.35
3.25 end

Here you see I invoked my “sine_tone” instrument with various parameters. The leading number and first parameter represent start and end times in seconds. The last two parameters map to the f and a variables of my instrument.

I wasn’t able to get the audio player plugin to work properly with the resulting .wav file nor was I able to get it to play with an itunes converted m4a…

UPDATE: I managed to get a proper mp3 toolchain going. I might describe that in more detail on Thursday. For now, here is a proper flash audio player

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.