> From: "Chalmers Jennifer" <JEC295@psy.soton.ac.uk>
> Date:Thu, 25 Apr 1996 09:35:33 GMT
>
> Would you mind summarising the last lecture on e mail? I understood
> most of it but I am afraid some of it went out of the window. You put
> up some pretty drawings of input/outputs but I didn't grasp what it was
> they were meant to explain.
I hope the reply to Liz answered part of your question, but here is a
recap of the lecture (I also suggest you read the chapter I
distributed).
Neural nets are the third in a series of candidates for what is going on
inside our heads to generate our behavioural capacity. The other two
candidates were computation (symbol manipulation) and analog processing
(e.g., "shadow" or image manipulation).
Neural nets are like the brain in certain respects. Both the brain and
nets have units that are interconnected. They may have positive or
negative connections. They pass activity from one unit to another.
Connections can get stronger or weaker. Some units code thing locally
(such as units that react to edges), and sometimes things are coded in a
distributed way, as a pattern of connections or activity, rather than
locally.
But there the resemblance stops. The neural nets that are used to model
cognition do not have axons or dendrites, action potentials or graded
potentials, synapses, neurotransmitters, glia, etc. Their resemblance to
the brain ("neurosimilitude") is very minimal. Nor is it necessary;
what we need is that they should be able to DO what we can do, for then
they provide a possible explanation for HOW we do it.
I described the perceptron in the reply to Lee, Liz. A backpropagation
network is like a perceptron with not just an input and an output layer
of units, but also intermediate or "hidden" layers. They work through
"supervised" learning: They receive an input. Whatever state they are
already in determines how that input propagates forward through the
connections of the hidden layer to the output layer. Suppose the output
is wrong. Then the net propagates the activity backwards, resetting the
connections that produced the wrong output to make it a little less
likely that they will do the same thing the next time they get the same
pattern of input. If the output is right, then the backpropagation
resets it to make it a little more likely that it will do that again the
next time.
By this means, a backprop net can learn patterns, many more than those
the perceptron could learn.
An unsupervised net gets no feedback about whether it is right or wrong;
it must sort its input patterns by self-organisation, which is done by
finding whatever structure there is in the input and enhancing it: If
some inputs are longer and some are shorter, the unsupervised net will
make the long ones even longer and the short ones even shorter, bringing
out the differences. This helps in bringing out boundaries and
contrasts.
Here is a question for you: What is more lifelike: supervised or
unsupervised learning? Are most of the categories and patterns we learn
based on finding and enhancing the contrasts that are there in the
input, or are they based on feedback about what is right and what is
wrong, and enhancing the contrast between those?
Neural nets are not symbol systems, though they can be simulated by
symbol systems (just as analog processing can be simulated by a symbol
system). When they are imlemented as real neural nets rather than
simulated by computers, then neural nets are a kind of analog processor,
one that uses parallel processing (many connections and activations
forming at the same time) and distributed processing (a pattern of
activity distributed across the whole system). Nets are especially
suited for learning, especially for learning patterns, learning to
categorise inputs. They are not as well suited for doing reasoning,
logical inference, calculation, or language as are symbol processors.
Fodor & Pylyshyn's critique of neural nets is based on what neural nets
cannot do, or cannot do as easily or as well as symbol systems: The
symbols in a symbol systems have arbitrary shapes, but they can be
interpreted as standing for something: A symbol system may have the
symbols "4 > 3" ("four is greater than three") in it, or "the cat is on
the mat." In a symbol system that has these symbols, you should also be
able to find the symbols "3 < 4" ("three is less than four"), or "the
mat is not on the cat" or "4 > 2". The symbols can be systematically
combined and recombined, and if you follow the symbol manipulation
rules it should all make sense.
But a neural net that has a node, or a pattern of activity, that we
would like to interpret as standing for "4 > 3" or "the cat is on the
mat" is unlikely to be decomposable into a bit that stands for "4" or
for ">" or for "on", so all those other systematic symbol combinations
that you get for free from a symbol system you don't have with a net.
For this reason, Fodor and Pylyshyn think that nets are not good
candidates for what is going on in our heads when we think, especially
when we think in language, or when we reason or calculate. (Pylyshyn is
the same Pylyshyn who challenged mental imagery and said it was all done
with symbols, in the Kosslyn paper.)
The truth is probably in the middle again, as it was with imagery:
Some things turned out to be done with analog processing, probably
because that was the easiest way to do it, rather than because symbols
couldn't have done it too. The same is true of neural nets: After all,
since neural nets can be simulated symbolically, whatever they can do,
symbols can do too. But sometimes it may be easier or more
straigtforward to do things by real parallel, distributed,
interconnected processing with separate physical units, rather than
simulating them serially through computation.
This archive was generated by hypermail 2b30 : Tue Feb 13 2001 - 16:23:39 GMT