This led to the motivation of learning distributed representations of words existing in low-dimensional space (Bengio et al., 2003).
Sobre maldição da dimensionalidade. Agora, o que seria representações distribuídas das palavras em espaços de menor dimensão? Isso me lembra de PCA e afins.
Thisistheideathateachinputtoasystemshouldberepresentedbymanyfeatures,andeachfeatureshouldbeinvolvedintherepresentationofmanypossibleinputs.
Sounds confusing, but think about the previous example of the picture with the woman and the different layers.
Think of the input as the picture, and it being represented by may features, like edges, shades, shape, etc.
Then those generalized concepts or features, being reused across different aspects of the photo, say for different parts where edges exist, or shapes. A very loose comparison is a reusable class that occurs throughout the program .