> From: "Lee, Liz" <EAL195@psy.soton.ac.uk>
> Date: Thu, 25 Apr 1996 09:08:36 GMT
>
> I do not understand exactly what substance a neural net takes - is it
> computer software?
Good question. You COULD make a neural net using real physical "nodes"
that were really physically interconnected to one another and could
activate one another in parallel (i.e., at the same time), strengthen
and weaken connections, and increasing and decreasing activations, etc.
That's the way to think of it, so if you are not interested in the
question of real vs. computer-simulated nets, skip the following bit,
which is an illustration of the power of symbols, the power of computer
simulation:
Most neural nets that are used and studied in cognitive science are
actually not real interconnected nodes, though; they are merely
SIMULATIONS of them, on a computer, just as the analog (image, shadow)
processing that Kosslyn was talking about was really a computer
simulation of analog processing rather than real analog processing.
There is nothing wrong with using a computer simulation instead of the
real thing, by the way, as long as the computer simulation encodes and
uses all the relevant properties of the real thing. What you find out
about one, will then be equally true about the other. You could have a
computer simulation of the planets revolving around the sun, and from
that you could make predictions and describe details about the real
sun and planets, because the simulation would be doing exactly the same
thing as the real thing (in all the relevant respects -- NOT their
physical location millions of miles apart, not their real size, for
none of that would fit in a computer!).
So a real net has a lot of connections and activations that go on in
parallel; in a simulation, they are all simulated serially, meaning that
nothing happens in parallel; it is all coded and happens one thing after
another. If you had one node activate two later nodes at exactly the
same time in the real net, in the simulation this would happen one
after the other, but the first node would be put on "hold" till the
second one also got activated, and then each of their activations would
be compared or combined still later.
Similarly, the activity patterns of a real net would be distributed in
real space; the North-West (NW) and South-East (SE) bits of the real
net might be simultaneously active, for example, while the rest of the
net was inactive. In a simulation, this would again not be parallel,
but serial (NW first, say, then put on hold, then SE), but it would
also not really be spatially distributed: things would not really be
happening in the northwest or southeast part of the computer. The
spatial location would, like the timing and the parallelness, merely be
encoded symbolically.
And the "substance" of the simulation would not be real nodes, but, as
you pointed out, software -- the programme the machine was running, in
other words, symbols and symbolic states.
Nothing is lost by thinking of a simulated neural net as one that is
simulated by doing paper and pencil calculations that exactly encode
what is active, and where. The net is not made out of paper and pencil
either.
But the important thing is that you CAN have a real parallel, distributed
physical net too, if you want to, and it really will do exactly what the
simulation only simulated. It's just easier to work with the simulation.
> How did the perceptron work?
This is fully explained in the chapter from the Best textbook, p. 249-50.
It is a two-layer network: An input layer of nodes and an output layer
of nodes. Each input node connects to each output node. Whenever the
perceptron gives a correct output in response to input, the strength of
the connections that led to it is increased; whenever the output is
wrong, the strength of the connections that led to it is decreased.
By this means, it can learn to recognise simple input patterns, for
example, the following "AND" problem:
Input/Output
01/no
00/yes
10/no
11/no
The rule it learns after some trial and error with feedback about
whether it was right or wrong is: Say yes if the first one is 0 AND
the second 1 is 0; say no otherwise.
Another pattern it can learn is (inclusive) "OR":
01/yes
00/yes
10/yes
11/no
Here the rule it learns is: Say yes if the first one is 0 OR the second
one is 0.
But here, as pointed out by Minsky in his critique of perceptrons, is a
pattern that the perceptron cannot learn, the one based on XOR or
"exclusive or":
01/yes
00/no
10/yes
11/no
The rule: Say yes if the first one is 0 OR the second is 1, but NOT
BOTH.
At the time the perceptron was first promoted by Rosenblatt and others,
it was thought to be the key to how the brain worked. Soon perceptrons
would be recognising all the patterns that people could recognise.
Unfortunately, as Minsky pointed out, this couldn't be true, because
a perceptron cannot even find a pattern that is based on XOR -- yet how
many patterns we know and recognise are based on being this or that but
not both!
In any case, the limitation has been overcome with many-layered nets,
which can not only handle XOR but can just about anything else.
> You didn't explain the diagram of 2 rows of circles - the middle 3
> circles of the bottom row did not appear to be linked to any others,
> why? I was less confused by the multi-layers idea and can see that it
> takes the form of a sort of decision making device.
I just didn't take the time to draw in all the connections! They're
meant to all be connected. The "action" is in the strength of the
connections, and that is determined by the inputs the net encounters,
and the feedback it gets about what is right and what is wrong
("supervised learning").
> As for Lorente de No, these "loops", having been substantiated, what
> do they show other than activity which carries on being passed in a
> circle rather than down a line until it peters out? Is original thought
> a new loop formation, or just a different route taken by the excitatory
> wave?
Good question. Hebb thought loops might be involved. No one knows yet.
But these days, apart from ordinary Hebb rule nets, and backpropagation
nets, there are also recurrent and re-entrant nets that have some more
of the properties that Hebb was the first to single out as possibly
underlying cognition.
This archive was generated by hypermail 2b30 : Tue Feb 13 2001 - 16:23:39 GMT