Monthly Archives: June 2012

Applying Machine Learning

I’ve successfully completed Andrew Ng’s Machine Learning class. I feel pretty spiffy that I can now say that I’ve implemented a Neural Network… though right now it’s just in Octave.

In his concluding video, Andrew asked that we go forth into the world and do cool things with machine learning… and I plan on doing exactly that. I can think of several interesting things that I can build in the context of my work at Topsy Labs… but it’s unlikely that any work context stuff will filter out in a way that the public can see directly.. so I plan on creating a few personal projects around machine learning.

The first thing that I want to do is go through the course material and actually implement most of it in Java.  For that purpose, I’ve checked in the start of said project into github as part of my incubator repo. I’ve affectionately named the sub-project “SkyNET” after the largely misunderstood AI from the Terminator Franchise.  The code itself won’t have specific spoilers from the exercises… that’d be bad form… but they will have brute force implementations of the algorithms taught in the course.

I think that actually implementing things from scratch, I’ll get a better handle of how and why some of this stuff works.  For this initial version of Skynet, I am not going to allow myself the luxury of third party libraries.  The point isn’t to create a well optimized machine learning system… It’s a learning exercise.  Part two of Skynet will be to re-implement the whole thing again in C++.  This time I’ll incorporate a bunch of well optimized libraries like Boost.

Those of you who want to troll through my Java code… here’s the direct link to Skynet.

Modeling Die Rolls

Modeling out die rolls seems pretty straight forward.  Let’s say that you wanted to figure out the distribution of sums. The brute force method for doing this looks like the following chunk of perl code:

sub dist {
    my ($count, $sides) = @_;
    my $state = [];
    for (my $i = 0; $i < $count; $i++) {
        $state->[$i] = 1;

    my $max = $count * $sides;
    my $result = [];
    while(1) {
       my $sum = 0;
       for (my $i = 0; $i < $count; $i++) {
          $sum += $state->[$i];

       last if $sum >= $max;
       for (my $i = 0; $i < $count; $i++) { 
           last if $state->[$i] [$i] = 1;
   return $result;

This seems to work well for distributions of same sided dice. What if I wanted to combine multiple types of dice? Say, 4x six sided dice and 8 4-sided dice… well, the brute force method starts to get pretty expensive.

I was reading this lecture on normal distributions for sums and came to the solution to my problem.  Instead of iterating over all sums for all dice, I could probably just add the normal curves for the die rolls.

How do you compute a normal distribution? It’s called a Gausian… check the Wikipedia page!  The code for a gausian given two params, mu and sigma look like so:

sub gausian {
    my ($x, $mu, $sigma) = @_;

    my $exp = - 1/2 * (($x - $mu) / $sigma) ** 2;
    return 1/($sigma * sqrt(2 * $pi)) * exp($exp);

Now I just need to figure out how to scale mu and sigma (mean, std deviation) to number of dice and the sides per die.  The lecture above seems to model 6 sided dice well with a mean of 3.5 x number of dice and 1.7 * sqrt(num of dice) but I need to expand that out to other side counts for this to work.

Simple IC Blender Model

It’s been a little while since I did stuff in Blender.  Here, I made a 16-pin IC.  I tried to make the model fairly generic so that I can easily do larger or smaller chips. If you want to check out the model, I have it checked into github:

Hello Seña dialog rough

I sat down with my friend Tomoko and did an initial reading of my script Hello Seña.

It’s been a while since I did anything related to Little Robots, so this was fun.  Naturally, this is not the final recording.  My Japanese is pretty obnoxious, but it’ll help a lot for storyboarding the scene.

Here’s a spliced together dialog.  You can hear a lot of background noise in this… I’ll have to figure out how to clean it up.  The recording itself was done on my iPhone4S.  Tomoko and I used Skype for the actual conversation.

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Weirdo mythology

I was considering the mythology of the Weirdo a moment and thought it might be fun to flesh out the character as a little kid.

It’s basically the start of 90% of all little kid stories.  You have a little kid who has an overactive imagination and next thing you know, all hell breaks loose.

I’m not really interested in coming up with a original story here, just a fun one… so starting with a cliche such as the weirdo being a little kid who happens to have a toy helmet is a safe premise to build from.

I find it unlikely any of the concept art stuff like the image to the left will ever actually make it into the game, but I need the drawing practice… and I wanted things fleshed out.