Neural Networks in Research: A top down approach.

It surprises me as I search the literature that there is so little research into Neural Networks that mimic nature, that mimic our brains. They just aren't fashionable I guess, who was it that made comments in the early days that had that effect? Marvin Minsky? Perhaps that is the reason. Then again perhaps I am looking in the wrong place!

Originally there were all those network that were nothing like a real brain, perceptrons, recurrent networks, Hopfield networks, all carefully organised, understood and solving one function, or making one decision. All these networks seem to be fully connected and perform in a logical way.

The animal brain, on the other hand is totally weird rather than logical, its not likely to be fully connected (though in truth we don't know) and it can do more than one thing. And it certainly seems to me that the mind, consciousness, thoughts, desires, the ability to recognise objects, people, danger, are an emergent property of the brain rather than things that can be found by logic and vivisection.

I have heard so many people say that we can't simulate a brain on a computer because brains are analogue not digital, (which isn't entirely true, but its not entirely true of computers either). These of course are the engineers and computer scientists. On the other hand if you talk to biologists then the problem is that computers are silicon based and not protoplasm based.

This whole thing is a bottom up approach. Attempting to solve fine details when there is a whole mountain to tackle.

Of course the real problem is that people are daunted by the mountain, so they put up the only obstacle that they are not daunted by, some detail.

It seems to me that aiming for a network that we can understand is on of the problems that is holding us back. Emergent properties can not be easily analysed, I mean we can't even define what intelligence is.

Although the working model for most of society seems to be, "whatever a computer can't do"!

The animal brain is the result of evolution, and given that we have a well established artificial evolutionary paradigm it seems to me natural to try and use artificial evolution to evolve artificial neural networks that we don't understand, and which produce *emergent behaviour* rather than compute a well defined function.


Well I think that is my rant over, back to searching the literature, and doing the programming. With any luck if I don't find much in the literature that is relevant to my topic it will be less to write up!

The Joys of Reading Other People's Code

I thought that it would make sense to read the code for NEAT and for Aria, but it turned out that it was probably not a good idea. The aim was that I would get a feel for what the code does and then I would have a change to write what I need. I didn't get that feel!

What I found was code where there is not enough comments to understand what is going on. Perhaps for the author there may have been enough, and he certainly would understand his own abbreviations (I assume) but there certainly was not enough for me.

OK, I think I probably put more comments than nearly all programmers, so probably I am biased any way. I think both internal and external documentation is very important, I try to use explicit names, although it is extremely unlikely that I will spell them correctly (most APIs don't include a spell checker!), which I am sure annoys lots of people!

Another problem was the indenting. To start with indentation was 2 spaces, which I find doesn't have the visual effect that I need. Secondly half of the indentation was all over the place, now I am assuming that this was the effect of importing the code into Visual Studio.

The next problem was the use of whitespace. In some places such as between identifiers and operators there was no whitespace, making whole lines of code one visually into word, not easily broken up by the eye. In other places there were loads of blank lines between every line of code. Sometimes there were even multiple blank lines between say a brace and the things outside the brace, so that it was not easy to see the connection that the braces made. As I write this I wonder if there were two coders with opposite styles involved, but that wouldn't really make sense. Perhaps importing affected whitespace as well as indenting.

I ended up spending days trying to sort out indenting, whitespace and matching braces, and didn't get much of a feel for the content at all. In frustration I changed indenting to a more Java style that a C++ style, which was probably, but now all the blank lines that are left are essential for comprehension, and probably some of those have gone too.

Finally the other thing that added to the obscurity of the code is the fact that functions, even after removal of blank lines, are often more that 200 lines long. My brain is simply not big enough to follow 200 lines of code in one lump. Clearly I am not a hacker! If a function goes much over one screen full its too much for me.

The length of the methods means that I am not at all sure that my attempt to find matching braces was correct. Does this close the if statement that started 20 lines ago, the switch statement that started 50 lines ago, the else that started 80 lines ago, the for loop that started 100 lines ago, or was it something before that???? Frankly all too often I gave up.

I don't have time to understand what this code does. I hope that I manage to find what I need to use, and don't miss anything too useful. If I use this code in the future I will definitely need to rewrite it.

Now when it comes to the ARIA code, well actually it wasn't much better. So at this stage, I don't know much, and I haven't achieved much!

I think I'll go and eat worms.

(Sorry that is a joke from the old country, I'm Australian now, but it just fits!)