> From: "HOLMES Sharon" <SHH@isvr.soton.ac.uk>
> Date: Mon, 13 May 1996 15:53:38 +0000
>
> I managed to get myself in a tizz this morning over Rumelhart's
> Perceptron model - what it did and didn't do with irregular and
> regular verbs. It would be extremely helpful if you could post to
> PY104, a description of how the perceptron model works.
Pinker's critique is in:
http://cogsci.soton.ac.uk/~harnad/Papers/Py104/pinker.conn.html
And replies are in:
http://cogsci.soton.ac.uk/~harnad/Papers/Py104/bates-1994.html
and
http://cogsci.soton.ac.uk/~harnad/Papers/Py104/bates_elman.html
In a nutshell, a perceptron is a very simple neural net with only an
input and output layer. It learns mainly by associating input patterns
to output patterns. If the input pattern is "give" and the output is
"gave", it can learn this, so whenever you give it "give" it gives you
"gave." Same for come/came, go/went, etc. These are all English
IRREGULAR transformations from present to past tense of verbs.
The perceptron learns them by simply memorising the associations.
When it is given live/lived, roam/roamed, and mow/mowed it can learn
those too, in exactly the same way, by memorising the cases, not
particularly "noticing" (except in a statistical sense) that they are
all following a simple, REGULAR rule (add "ed").
Now give it a new one that it has never seen: "grow": It might even,
for statistical reasons, respond "growed" (and then the
overenthusiastic observer might want to say "aha, it made the same
mistake that children make), but unfortunately it may just as well
respond "grent," which children never say. And in general, unlike
children, it never gets very good on the regulars; the only thing it
does unfailingly well is the irregulars (and regulars) that it has
already seen.
That's why Pinker & Prince concluded that memorising special cases by
rote association was all that nets were good at, hence they were not
suitable models for what's going on in the mind., because so much of
what the mind does is rule-based, rather than just being based on
associations and memory. So symbol systems, according to P & P, are
better models for what's going on in the mind than nets, at least for
the vast majority of cases, where a rule must be known or learnt, and
not just a set of associations memorised.
What is the answer to this critique? The perceptron is just a simple
two-layered pattern associator, but add just one more layer, a hidden
layer, and the net can learn rules (simple ones, but rules nonetheless);
and with enough layers and units, and enough data, nets could learn any
rule. So nets may still be good models for what's going on in our minds
after all.
Minsky's critique had been that perceptrons could not do XOR problems,
but again, adding more layers remedied this problem too.
Of course, there are other problems with nets: Fodor & Pylyshyn's
(1988) critique that nets do not have the systematic combinatory power
of symbol systems is not so easily answered. You may be able to train
nets to recognise categories by detecting features, based on ruleful
combinations of features (red OR big AND NOT striped), but it's not so
easy to get nets to do the kinds of things that symbol systems do quite
naturally: In a symbol system, if there is symbol state: "X is red or
big and not striped" then there is also one for "Y is stiped and not
big or red" and so on. With a net, each of these logical combinations
would have to be trained up separately, on its own set of input
patterns. You could go on forever doing that, whereas the symbol system
has them all for free.
So maybe symbol systems are better models of what's going on in our
heads.
But then there's the symbol rounding problem...
http://cogsci.soton.ac.uk/~harnad/Papers/Harnad/harnad91.otherminds.html
----Fodor, J.A., & Pylyshyn, Z.W. (1988). Connectionism and cognitive architecture: A critical analysis. In S. Pinker & J. Mehler (Eds.), Connections and Symbols. Cambridge, MA: MIT Press/Bradford Books. Pp. 3-71.
This archive was generated by hypermail 2b30 : Tue Feb 13 2001 - 16:23:41 GMT