Category: Artificial Intelligence


Algorithm

Current algorithm used to extract sparse features as shown in the previous post.

Extracted sparse features

My current online feature abstraction seems to work fine now. After feeding the system a 30 minute video once it abstracts features (or “components”) of the input by changing each virtual neurons synapses (right plot in image). This is done in a way which yield a sparse and quasi-binary neural response to the input (“sparse” to decorrelate the mixed signals and “quasi-binary” to be tolerant to noise).

The reconstruction shows that all essential information is preserved and can be reconstructed from the sparse neural response.

In science fiction we often see highly sophisticated self-learning robots, yet most of them completely lack emotions. I believe that even theoretically it won’t be possible to create a robot which can learn to behave in a physical world without having some sort of emotions. But I’m sure emotions aren’t some mystical force, so it can be explained somehow.

Here’s my attempt:
I believe emotions cannot all be treated the same way as some emotions are built on top of other, more low-level emotions. At the lowest level there are only two emotions necessary to give a specific situation informative value and basic judgement: positive/good and negative/bad.

Positive emotions will modify the behaviour in order to increase the chance to reproduce (steering towards) the situation and a negative emotion will increase the chance to avoid (steering away from) the situation.
At first, what is considered good and bad is genetically pre-wired in an individual to maximize chance of survival and reproduction and consists of basic concepts like “physical pain incl hunger etc is bad” or “curiosity, family-bond and sexual attraction etc is good”.
This basic system is implemented in all learning organisms, probably even down to bacteria which learn to avoid poisenous substances.
This also means that any learning system will only learn what is emotionally significant, which is necessary to give importance to the nearly unlimited number of possible learning-options and situations. This means there can be no learning system (including future robots) without these low-level emotions.

Of course our brains work on a bit more complex level: As newborns will behave after the above simple principle, adults are not only experiencing these two basic emotions and also act emotionally not only to the pre-wired situations. The latter is due to the fact, that the brain learns to predict future situations from current and past situations and associates to those the emotional value of the predicted future situation. On an abstract level this means that I’ve associated “buying groceries” to “being good when hungry”, because I’ve learned the connection that I can buy food there and food is good against hunger. So situations “on the way” to a good or bad emotion will be associated with these as well. This prediction of the future will also cause one new emotion to arise: Surprise. It is quite a strong, but usually short emotion and its purpose is to correct a wrong prediction. If I have learned that a ball always rolls down a slope and now I make the observation that it doesn’t I’d be quite surprised. That’s usually how jokes work, btw – guide prediction in a certain way and then counter with something quite opposing.

So what about all the other emotions like joy, fear, anger, grief and so on ? I believe they arise from the prediction of certain low-level emotions and are therefore emergent from these low-level emotions. For example joy emerges when we get into a situation in which we predict something good to happen. Opposingly fear emerges when we predict something bad to happen. Anger will emerge when we predict something good to happen, but realize that we cannot reach that situation because of some obstacle. Grief will emerge when we realize that something good is not achievable anymore (e.g. a loved person died). Also the brain will create emotion-specific behavioural pathways which help us dealing best with a situation when being in a certain emotional state.

All emotions are transitional, which means that we cannot sustain any one emotion for long periods of time. This is necessary to avoid getting stuck in any emotion, though it still happens to some people in which this system is malfunctioning or which have been traumatized (sort of over-learning due to an excessive emotion).

This is part of my “artificial perception” project.

An image (16×16 pixels) is generated which consists of 8 horizontal and 8 vertical white bars with each being either shown or hidden by chance. This means there is a total of 2^16=65536 patterns possible. Additionally the generated image is degraded by adding 50% noise to make recognition harder (this ultimately increases the number of possible patterns to infinite).

(A) shows a small subset of these input patterns.

Now these patterns are presented one-by-one (no batch processing) to a system consisting of 20 artificial neurons (amount is arbitrarily choosable) and each neuron updates its synapses by my new learning-rule.
The idea is that the system “learns to understand” the pattern-generating process and instead of trying to remember all possible patterns (65536 without noise and close to infinity for the noisy ones) which is completely unfeasable due to size constraints it extracts each bar seperately (even tho they are almost never shown in isolation, but always in combination with other bars). It does so as every pattern experienced so far can be reconstructed from these (2*8) separate bars.

(B) shows that 4 of the 20 neurons will remain unspecialized while the other 16 will specialize towards one of the 2*8 bars.

Also as you can see each neuron’s specialization in (B) is mainly free of noise (and therefore the whole percept) even tho a great amount of noise was inherent in each presented pattern.

This results in a 16-bit code which is the optimal solution to represent all patterns in a compact manner.

Computational complexity is O(N²) where N is the number of neurons. Memory complexity is just O(N).
Complexity is NOT dependent on the amount of patterns shown (or possible).
This should be one of the fastest online learning methods for this test without storing or processing previously experienced patterns.