LING 385: Lecture 8 | Notes by Cole Gawin

What does a Neural Network (NN) try to do?

three psychological skills that an NN tries to emulate mathematically
1. ability of an animal to have knowledge about its environment
2. ability of an animal to recognize things in its environment, using its knowledge
3. ability of an animal to learn about its environments, thereby a) acquiring knowledge about it and b) being able to recognize those objects
it is hard to imagine animal life without these three psychological abilities
the secret power of NN’s is that they are able to capture these three psychological skills using a little more than arithmetic

Similar means: +’s correspond to +’s, and -’s to -’s, mostly.
Dissimilar means: Some +’s correspond to +’s, some -’s to -’s, but some +’s correspond to -’s

First example: A NN that has knowledge about faces, its environment, can recognize the faces of different people, assigning them their names
a face is a table of numbers, the brightness/color of pixels
NN’s represent a face as a list or vector of numbers
we can get a list by taking the first row of numbers, following it by the second row of numbers, etc
each of your friends then has a vector of numbers representing their face
if an NN knows the names of your friends, then if we give it a vector of one of your friends as an input, it will output some number that represents your friend’s name
recognition: NN is a function that takes an environmental input vector x and spits out a vector memory output

an object in the environment is represented by an input vector x
knowledge is represented by a vector W
recognition is represented by taking the inner product between the W vector and the x vector: <W , x>
squashing function: if <W , x> < 0, output is 0; else, output is 1
hence, recognition is a function that uses knowledge, inner-producting its W with input x, and then using a squashing function to output 0 or 1